-- Logs begin at Thu 2022-06-30 17:14:44 UTC, end at Thu 2022-06-30 18:53:58 UTC. -- Jun 30 18:13:44.331584 dv-cinder-803785-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 30 18:13:44.345471 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 30 18:13:44.348984 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 30 18:13:44 2022] *** Jun 30 18:13:44.348984 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 30 18:13:44.348984 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 30 18:13:44.348984 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: nodename: dv-cinder-803785-1-iscsi Jun 30 18:13:44.348984 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: machine: x86_64 Jun 30 18:13:44.348984 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: clock source: unix Jun 30 18:13:44.348984 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: pcre jit disabled Jun 30 18:13:44.348984 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: detected number of CPU cores: 4 Jun 30 18:13:44.348984 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: current working directory: / Jun 30 18:13:44.348984 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: detected binary path: /usr/bin/uwsgi-core Jun 30 18:13:44.349536 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: your processes number limit is 47680 Jun 30 18:13:44.349536 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: your memory page size is 4096 bytes Jun 30 18:13:44.349536 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: detected max file descriptor number: 2048 Jun 30 18:13:44.349536 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: lock engine: pthread robust mutexes Jun 30 18:13:44.349536 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: thunder lock: enabled Jun 30 18:13:44.349536 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 30 18:13:44.349873 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 30 18:13:44.427213 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: Python main interpreter initialized at 0x55f6c37389c0 Jun 30 18:13:44.427213 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: python threads support enabled Jun 30 18:13:44.427213 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: your server socket listen backlog is limited to 100 connections Jun 30 18:13:44.427213 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: your mercy for graceful operations on workers is 90 seconds Jun 30 18:13:44.427687 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: mapped 403077 bytes (393 KB) for 2 cores Jun 30 18:13:44.427863 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: *** Operational MODE: preforking *** Jun 30 18:13:44.428382 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: *** uWSGI is running in multiple interpreter mode *** Jun 30 18:13:44.428382 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: spawned uWSGI master process (pid: 80197) Jun 30 18:13:44.428616 dv-cinder-803785-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 30 18:13:44.429626 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: spawned uWSGI worker 1 (pid: 80206, cores: 1) Jun 30 18:13:44.435893 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: spawned uWSGI worker 2 (pid: 80207, cores: 1) Jun 30 18:13:44.435893 dv-cinder-803785-1-iscsi devstack@keystone.service[80197]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 30 18:13:50.030249 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 18:13:50.030249 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: from cryptography.utils import int_from_bytes Jun 30 18:13:50.030931 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 18:13:50.030931 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: from cryptography.utils import int_from_bytes Jun 30 18:13:50.046397 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 18:13:50.046397 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: from cryptography.utils import int_from_bytes Jun 30 18:13:50.047020 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 18:13:50.047020 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: from cryptography.utils import int_from_bytes Jun 30 18:13:50.080855 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 30 18:13:50.081058 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 30 18:13:50.081225 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] command line args: [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 30 18:13:50.081500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 30 18:13:50.081704 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ================================================================================ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 30 18:13:50.081965 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] admin_token = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.082216 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] config_dir = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.082405 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.082590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] config_source = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.082773 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] debug = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.082974 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.083350 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.083537 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.083798 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.084055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.084293 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.084489 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_config_append = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.084670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.084878 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_dir = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.085054 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_file = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.085301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.085668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.085784 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.086042 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.086231 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.086395 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.086646 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.086897 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.087134 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.087373 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.087607 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.087844 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.088097 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.088344 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.088624 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.089870 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.090057 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] publish_errors = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.090301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.090539 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.090723 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.090975 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.091216 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.091454 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] standard_threads = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.091689 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.091935 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.092114 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.092294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_journal = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.092471 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_json = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.092649 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_stderr = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.092827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] use_syslog = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.093062 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.093325 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.093568 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.093858 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.094105 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.094357 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.094602 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.094855 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.095103 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.095358 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.095610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.095744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 30 18:13:50.095876 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.096009 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 30 18:13:50.096143 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.096281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] command line args: [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 30 18:13:50.096409 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.096558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 30 18:13:50.096558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ================================================================================ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 30 18:13:50.096799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.096925 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] admin_token = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.097060 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.097193 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] config_dir = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.097327 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.097454 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.097615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.097761 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] config_source = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.097900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.098028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] debug = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.098028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.098315 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.098472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.098633 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.098772 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.098901 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.099049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.099229 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.099359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.099359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.099608 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.099743 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_config_append = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.099871 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.100018 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.100018 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_dir = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.100264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.100478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_file = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.100618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.100754 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.100894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.101023 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.101156 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.101285 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.101411 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.101537 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.101727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.101868 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.101997 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.102125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.102254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.102394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.102529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.102657 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.102787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.102914 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.103041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.103176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.103305 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.103432 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.103569 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.103697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.103825 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.103963 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.104092 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.104220 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.104418 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.104590 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.104722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.104722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.104980 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.105115 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.105243 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] publish_errors = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.105459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.105640 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.105794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.105924 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.106061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.106196 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.106196 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.106525 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.106659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.106795 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.106925 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] standard_threads = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.107053 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.107189 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.107325 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.107466 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.107594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.107722 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.107850 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.107988 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_journal = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.107988 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_json = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.108237 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.108365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_stderr = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.108493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.108620 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] use_syslog = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.108758 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.108901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 18:13:50.109034 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.109172 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.109300 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.109428 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.109563 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.109739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.109877 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.110004 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.110139 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.110267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.110403 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.110542 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.110675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.110809 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.110937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.111083 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.111293 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.111430 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.111565 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.111694 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.111821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.111948 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.112085 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.112216 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.112343 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.112470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.112597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.112731 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.112881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.113010 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.113137 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.113265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.113393 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.113520 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.113684 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.113815 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.113944 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.114071 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.114205 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.114343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.114472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.114600 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.114884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.115019 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.115148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.115296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.115434 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.115569 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.115698 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.115825 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.115985 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.116128 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.116258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.116396 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.116526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.116666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.116794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.116931 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.117061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.117190 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.117318 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.117454 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.117643 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.117794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.117938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.118069 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.118210 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.118340 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.118470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.118610 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.118739 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.118889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.119018 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.119147 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.119276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.119405 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.119540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.119677 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.119807 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.120462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.120607 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.120743 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.120893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.121024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.121154 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.121282 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.121411 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.121540 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.121707 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.121839 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.121968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.122096 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.122225 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.122360 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.122498 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.122628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.122768 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.122897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.123026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.external = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.123154 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.123291 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.123421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.123549 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.123679 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.123809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.123946 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.124082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.password = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.124212 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.124341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] auth.token = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.124479 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.124608 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.124737 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.124874 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.125004 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.125133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.125262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.125391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.125519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.125687 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.125821 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.125957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.126087 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.126215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.126354 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.126578 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.126720 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.126859 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.126998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.127129 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.127259 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.127388 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.127517 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.127653 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.127783 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.127783 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.128029 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.128029 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.128284 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.128284 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.128542 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.128671 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.128799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.128928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.129065 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.129211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.129342 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.129471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.129663 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.129663 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.129936 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.130078 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.130208 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.130344 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.130473 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.130603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.130741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.130870 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.130999 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.131159 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.131298 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.131430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.131560 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.131691 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.131839 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.131977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.132108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.132108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.132354 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.132354 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.132608 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.132740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.132872 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.133002 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.133132 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.133132 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.133391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.133523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.133709 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.133843 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.133985 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.134117 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.134257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.134388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.134525 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.134656 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.134787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.134917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.135056 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.135188 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.135318 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.135448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.135647 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.136292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.136433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.136565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.136696 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.136826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.136956 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.137086 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.137225 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.137363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.137495 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.137658 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.137797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.137937 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.138077 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.138216 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.138348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.138487 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.external = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.138618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.138749 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.138893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.139025 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.139167 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.139299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.139429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.139561 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.password = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.139715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.139847 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] auth.token = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.139977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.140107 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.140236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.140366 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.140505 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.140637 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.140767 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.140903 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.141034 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.141163 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.141302 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.141433 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.141639 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.141781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.141912 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.142049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.142261 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.142394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.142524 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.142654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.142785 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.142915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.143057 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.143189 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.143319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.143449 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.143595 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.143727 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.143866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.143997 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.144128 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.144257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.144387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.144525 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.144657 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.144787 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.144917 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.145046 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.145177 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.145329 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.145460 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.145613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.145753 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.145884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.146014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.146154 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.146380 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.146516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.146652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.146784 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.146914 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.147052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.147183 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.147323 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.147453 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.147583 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.147712 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.147851 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.147984 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.148120 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.148250 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.148250 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.148495 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.148633 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.148764 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.148894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.149031 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.149031 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.149291 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.149291 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.149546 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.149732 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.149865 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.149995 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.150124 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.150265 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.150397 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.150535 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.150666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.150796 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.150936 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.151075 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.151250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.151383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.151512 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.151641 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.151771 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.151908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.152040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.152170 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.152300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.152429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.152559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.152696 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.152841 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.152841 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.153087 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.153217 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.153353 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.153353 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.153631 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.153778 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.153910 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.154040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.154172 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.154312 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.154443 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.154584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.154714 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.154850 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.154982 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.155119 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.155252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.155390 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.155523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.155653 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.155784 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.155784 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.156039 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.156170 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.156302 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.156450 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.156581 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.156720 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.156852 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.156852 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.157096 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.157226 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.157365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.157495 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.157662 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.157799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.157931 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.158061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.158202 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.158344 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.158475 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.158604 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.158734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.158734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.158997 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.159128 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.159258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.159389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.159519 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.159649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.159786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.159917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.160057 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.160187 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.160317 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.160317 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.160572 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.160702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.160832 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.160961 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.161090 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.161219 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] role.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.161357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.161488 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] role.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.161640 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.161776 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.161921 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.162052 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.162193 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.162325 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.162455 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.162586 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.162716 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.162716 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.162973 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.163106 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.163236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.163366 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.163495 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.163636 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.163776 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.163907 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.164038 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.164168 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.164298 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.164427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.164565 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.164696 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.164826 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.164956 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.165084 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.165214 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.165352 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.165493 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.165493 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.165769 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.165899 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.166029 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.166170 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.166301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.166432 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.166562 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.166691 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.166888 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.167036 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.167168 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.167310 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.167449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.167581 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.167722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.167930 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.168063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.168193 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.168323 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.168453 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.168590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.168722 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.168860 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.168994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.169125 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.169266 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.169405 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.169536 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.169732 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.169905 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.170043 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.170174 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.170586 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.170722 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.170852 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.170983 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.171112 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.171242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.171390 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.171521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.171651 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.171781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.171910 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.172041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.172178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.172309 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.172439 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.172569 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.172699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.172829 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.172973 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.173105 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.173245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.173376 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.173506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.173661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.173806 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.173937 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.174068 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.174197 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.174353 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.174485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.174623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.174754 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.174885 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.175015 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.175156 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.175286 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.175424 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.175555 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.175685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.175814 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.175944 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.176074 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.176213 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.176352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] role.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.176492 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.176624 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] role.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.176624 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.176871 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.176871 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.177135 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.177267 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.177397 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.177527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.177727 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.177859 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.178001 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.178133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.178263 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.178394 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.178524 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.178653 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.178808 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.178943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.179074 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.179074 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.179321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.179451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.179589 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.179721 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.179853 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.179983 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.180114 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.180243 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.180384 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.180384 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.180642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.180773 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.180902 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.181032 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.181169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.181302 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80206) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 30 18:13:50.181434 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.181434 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.181434 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.181434 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.181434 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.181434 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.181434 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.181434 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.181434 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.182166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.182166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.182166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.182166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.182166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.182166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.182166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.182166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.182166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.183416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.183416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.183416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.183416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.183416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.183416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.183416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.183416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.183416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.183858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.183858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.183858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.183858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.183858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.183858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.183858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.183858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.183858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.184345 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.184345 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.184345 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.184345 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.184345 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.184345 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.184345 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.184345 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.184345 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.184798 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.184798 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.184798 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.184798 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.184798 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.184798 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.184798 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.184798 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 18:13:50.184798 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80207) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 30 18:13:50.201950 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.202183 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.202417 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.209095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.209335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.209567 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.211086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.218472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.239657 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.239889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.244444 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.244683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.374377 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.383723 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.426004 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.426246 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.426739 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.426974 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.493841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.494073 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.494301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.494527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.494748 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.495001 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.495374 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.495616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.495843 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.496068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.496289 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.496508 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.521411 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.521658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.521753 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.521889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.521974 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.522117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.522229 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.522340 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.522426 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.522559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.522642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.522775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.522859 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.522989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.534749 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.535224 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.548547 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.548782 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.549011 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.549099 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.549242 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.549336 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.549462 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.549550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.549771 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.549882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.549967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80206) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.550061 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.550142 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.550359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80207) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 18:13:50.568025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.568234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 18:13:50.568349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.568479 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 18:13:50.568565 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.568771 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 18:13:50.568856 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.569007 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 18:13:50.569100 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.569276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.569360 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.569533 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.569639 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.569822 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.569906 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.570076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.581206 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.581479 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.581725 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.581831 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.582006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.582095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.582279 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.582566 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.590567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.591037 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.593902 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.594180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.594364 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.594469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.594644 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.594729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.594928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.595023 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.595250 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.595340 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.595521 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.595602 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.595795 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.595875 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.596068 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.596150 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.596344 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.596424 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.596612 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.596699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.596885 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.597150 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.624613 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.625070 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.626226 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.626508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.626750 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.627028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.631881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.632160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.632356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.632442 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.632640 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.632920 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.640478 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.640758 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.641137 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.641427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.645275 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.645556 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.645860 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.646120 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.646215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.646407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.646501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.646676 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.646772 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.646943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.647026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.647214 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.647295 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.647487 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.647757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.648027 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.666669 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.667217 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.667647 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.667867 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.672063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.672712 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.675727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.676022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.676423 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.676718 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.681206 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.681775 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.684938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.685878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.688642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.688940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.689226 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.689522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.689721 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.689842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.690022 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.690135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.690307 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.690591 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.690869 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.691145 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.709824 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.710123 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.710391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.710654 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.710741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.712583 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.712866 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.713136 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.719221 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.720947 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.721226 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.721714 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.723443 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.723721 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.724471 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.724753 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.725033 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.725314 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.725625 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.726838 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.727122 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.727404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.727692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.727972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.738995 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.739273 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.739563 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.739844 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.740128 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.740402 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.740685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.741689 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.741981 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.742264 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.742548 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.742835 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.743110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.743385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.760150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.761947 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.763969 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.765218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.767329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.768564 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.768924 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.770180 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.770755 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.771004 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.771288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.771818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.772102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.772401 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.779417 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.780426 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.781029 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.782152 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.784311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.784609 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.784891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.785170 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.785360 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.785677 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.785964 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.786258 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.794576 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.794856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.795145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.795852 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.796140 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.796420 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.803303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80206) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.803582 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.803861 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.804136 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.804315 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80207) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 18:13:50.804399 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.804595 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.804679 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.804872 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.804955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.805146 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.805292 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.805414 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.805595 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.805718 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.805875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.805993 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.806162 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.806279 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.806432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.806556 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.806702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.806832 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.806967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80206) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.807101 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.807417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.807864 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.808267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80207) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 18:13:50.846236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f6c37389c0 pid: 80206 (default app) Jun 30 18:13:50.847378 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f6c37389c0 pid: 80207 (default app) Jun 30 18:13:50.849770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694deaf6-7f23-45c1-ab5f-9a0a69193b89 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:13:50.850106 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694deaf6-7f23-45c1-ab5f-9a0a69193b89 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:13:50.850361 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694deaf6-7f23-45c1-ab5f-9a0a69193b89 None None] PATH_INFO: `/v3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:13:50.854820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Thu Jun 30 18:13:50 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:13:58.841015 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a882ba8-1dfc-4008-aaa7-00f37e3e4ee2 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:13:58.841634 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a882ba8-1dfc-4008-aaa7-00f37e3e4ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:13:58.841634 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a882ba8-1dfc-4008-aaa7-00f37e3e4ee2 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:13:58.845911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1/2] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:13:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:13:58.852005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e2cfc-b326-44ad-889b-cb7abdea3036 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:13:58.852461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e2cfc-b326-44ad-889b-cb7abdea3036 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:13:58.852562 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e2cfc-b326-44ad-889b-cb7abdea3036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:13:58.923812 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: INFO dbcounter [None req-2e2e2cfc-b326-44ad-889b-cb7abdea3036 None None] Registered counter for database keystone Jun 30 18:13:58.945379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG oslo_db.sqlalchemy.engines [None req-2e2e2cfc-b326-44ad-889b-cb7abdea3036 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80206) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 30 18:13:58.951958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writer thread running {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 30 18:13:59.127811 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG passlib.handlers.bcrypt [None req-2e2e2cfc-b326-44ad-889b-cb7abdea3036 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80206) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 30 18:13:59.128148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG passlib.handlers.bcrypt [None req-2e2e2cfc-b326-44ad-889b-cb7abdea3036 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80206) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 30 18:13:59.183738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2e2e2cfc-b326-44ad-889b-cb7abdea3036 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:13:59.185369 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2e2e2cfc-b326-44ad-889b-cb7abdea3036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:13:59.315186 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 2/3] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:13:58 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 30 18:13:59.503595 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a882ba8-1dfc-4008-aaa7-00f37e3e4ee2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:13:59.576265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: INFO dbcounter [None req-1a882ba8-1dfc-4008-aaa7-00f37e3e4ee2 None None] Registered counter for database keystone Jun 30 18:13:59.596316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG oslo_db.sqlalchemy.engines [None req-1a882ba8-1dfc-4008-aaa7-00f37e3e4ee2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80207) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 30 18:13:59.602698 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writer thread running {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 30 18:13:59.720565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a882ba8-1dfc-4008-aaa7-00f37e3e4ee2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:13:59.724860 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117f6ce-c5ea-4dd8-91ba-656bc73d5ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:13:59.725512 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117f6ce-c5ea-4dd8-91ba-656bc73d5ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:13:59.725826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117f6ce-c5ea-4dd8-91ba-656bc73d5ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:13:59.726102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117f6ce-c5ea-4dd8-91ba-656bc73d5ae3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:13:59.734397 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a117f6ce-c5ea-4dd8-91ba-656bc73d5ae3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:13:59.785717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a117f6ce-c5ea-4dd8-91ba-656bc73d5ae3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 30 18:13:59.785986 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a117f6ce-c5ea-4dd8-91ba-656bc73d5ae3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:13:59.796935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 2/4] 10.222.0.33 () {58 vars in 1324 bytes} [Thu Jun 30 18:13:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 18:13:59.803085 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e2cfc-b326-44ad-889b-cb7abdea3036 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:13:59.818279 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e2cfc-b326-44ad-889b-cb7abdea3036 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:13:59.823023 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506d0fc-2adb-45ab-becb-ee3ae2253f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:13:59.824162 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506d0fc-2adb-45ab-becb-ee3ae2253f2a admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:13:59.824426 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506d0fc-2adb-45ab-becb-ee3ae2253f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:13:59.824701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506d0fc-2adb-45ab-becb-ee3ae2253f2a admin admin] PATH_INFO: `/v3/projects/6ed5e6c375de4a6186ba026da89db56b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:13:59.828860 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506d0fc-2adb-45ab-becb-ee3ae2253f2a admin admin] RBAC: Authorizing `identity:get_project(project_id=6ed5e6c375de4a6186ba026da89db56b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:13:59.880118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4506d0fc-2adb-45ab-becb-ee3ae2253f2a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 30 18:13:59.880747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506d0fc-2adb-45ab-becb-ee3ae2253f2a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:13:59.885133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 3/5] 10.222.0.33 () {56 vars in 1207 bytes} [Thu Jun 30 18:13:59 2022] GET /identity/v3/projects/6ed5e6c375de4a6186ba026da89db56b => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:01.926558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ea018-b439-4f60-b3b3-93d9a61ec31e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:01.927109 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ea018-b439-4f60-b3b3-93d9a61ec31e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:01.927109 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ea018-b439-4f60-b3b3-93d9a61ec31e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:01.931242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 3/6] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:14:01.938036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a691c8e-3968-40a5-a33f-7dc9a05343b3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:01.938503 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a691c8e-3968-40a5-a33f-7dc9a05343b3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:01.938599 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a691c8e-3968-40a5-a33f-7dc9a05343b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:01.957445 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4a691c8e-3968-40a5-a33f-7dc9a05343b3 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:01.959246 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4a691c8e-3968-40a5-a33f-7dc9a05343b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:01.967749 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 4/7] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 30 18:14:01.992298 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ea018-b439-4f60-b3b3-93d9a61ec31e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:02.005772 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ea018-b439-4f60-b3b3-93d9a61ec31e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:02.009960 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee4974-ab9e-4288-b9aa-5e7ab3644818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:02.010598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee4974-ab9e-4288-b9aa-5e7ab3644818 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:02.010881 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee4974-ab9e-4288-b9aa-5e7ab3644818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:02.011158 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee4974-ab9e-4288-b9aa-5e7ab3644818 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:02.019494 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bee4974-ab9e-4288-b9aa-5e7ab3644818 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:02.021459 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bee4974-ab9e-4288-b9aa-5e7ab3644818 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:02.031010 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 4/8] 10.222.0.33 () {58 vars in 1324 bytes} [Thu Jun 30 18:14:01 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 18:14:02.036955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a691c8e-3968-40a5-a33f-7dc9a05343b3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:02.047224 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a691c8e-3968-40a5-a33f-7dc9a05343b3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:02.051710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f486a89-c6e0-460f-aa4e-8afea94cb8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:02.052583 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f486a89-c6e0-460f-aa4e-8afea94cb8a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:02.052859 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f486a89-c6e0-460f-aa4e-8afea94cb8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:02.053132 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f486a89-c6e0-460f-aa4e-8afea94cb8a3 admin admin] PATH_INFO: `/v3/users/de745ad7f2ed48ffbb043529099e53f0` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:02.054574 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f486a89-c6e0-460f-aa4e-8afea94cb8a3 admin admin] RBAC: Authorizing `identity:get_user(user_id=de745ad7f2ed48ffbb043529099e53f0)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:02.056421 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f486a89-c6e0-460f-aa4e-8afea94cb8a3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:02.058797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 5/9] 10.222.0.33 () {56 vars in 1198 bytes} [Thu Jun 30 18:14:02 2022] GET /identity/v3/users/de745ad7f2ed48ffbb043529099e53f0 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:04.195876 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801934a-2517-4245-a2d0-f213e1deaec2 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:04.196523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801934a-2517-4245-a2d0-f213e1deaec2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:04.196523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801934a-2517-4245-a2d0-f213e1deaec2 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:04.198716 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 5/10] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:04.205415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8722394-859e-4982-8a97-387ee7b21b16 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:04.205990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8722394-859e-4982-8a97-387ee7b21b16 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:04.205990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8722394-859e-4982-8a97-387ee7b21b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:04.212188 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e440c-86e0-4a3c-9713-2dc7959dac1b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:04.212557 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e440c-86e0-4a3c-9713-2dc7959dac1b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:04.213314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e440c-86e0-4a3c-9713-2dc7959dac1b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:04.214478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 6/11] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:04.220615 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0e2c5-8f96-4e77-942c-f55554d98edd None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:04.221085 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0e2c5-8f96-4e77-942c-f55554d98edd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:04.221524 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0e2c5-8f96-4e77-942c-f55554d98edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:04.226040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d8722394-859e-4982-8a97-387ee7b21b16 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:04.227250 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d8722394-859e-4982-8a97-387ee7b21b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:04.279299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG passlib.handlers.bcrypt [None req-d1c0e2c5-8f96-4e77-942c-f55554d98edd None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80207) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 30 18:14:04.279642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG passlib.handlers.bcrypt [None req-d1c0e2c5-8f96-4e77-942c-f55554d98edd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80207) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 30 18:14:04.313276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 6/12] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 18:14:04.326211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d1c0e2c5-8f96-4e77-942c-f55554d98edd None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:04.327864 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d1c0e2c5-8f96-4e77-942c-f55554d98edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:04.335571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8722394-859e-4982-8a97-387ee7b21b16 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:04.361167 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:04.428233 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 7/13] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:04.444926 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8722394-859e-4982-8a97-387ee7b21b16 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:04.451840 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0e2c5-8f96-4e77-942c-f55554d98edd None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:04.482400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c756e213-496d-43b0-8295-6ad1b7e92306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:04.483065 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756e213-496d-43b0-8295-6ad1b7e92306 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:04.483355 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756e213-496d-43b0-8295-6ad1b7e92306 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:04.483666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756e213-496d-43b0-8295-6ad1b7e92306 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:04.555477 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c756e213-496d-43b0-8295-6ad1b7e92306 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:04.557035 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c756e213-496d-43b0-8295-6ad1b7e92306 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:04.561224 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0e2c5-8f96-4e77-942c-f55554d98edd None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:04.563734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:04.582633 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:04.606884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0012c16-3274-4d75-87f1-eb5d46adc6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:04.607999 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0012c16-3274-4d75-87f1-eb5d46adc6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:04.608293 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0012c16-3274-4d75-87f1-eb5d46adc6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:04.608579 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0012c16-3274-4d75-87f1-eb5d46adc6d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:04.623080 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0012c16-3274-4d75-87f1-eb5d46adc6d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:04.634362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-c0012c16-3274-4d75-87f1-eb5d46adc6d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:14:04.634362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:04.634362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:04.634362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:04.634362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:04.634362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:04.634362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:04.634362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:04.634362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:04.634362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:04.634362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:04.634362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:04.634362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:04.634362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:04.634362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:04.634362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:04.635357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:04.635357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:04.635357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:04.635357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:04.635357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:04.635357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:04.635357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:04.635357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:04.635357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:04.635357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:04.635357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:04.635357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:04.635357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:04.635357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:04.635357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:04.635357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:04.636185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:04.636185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:04.636185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:04.636185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:04.636185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:04.636185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:04.636185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:04.636185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:04.636185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:04.636185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:04.636185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:14:04.636185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:14:04.636185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 8/14] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:14:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:14:04.641772 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0012c16-3274-4d75-87f1-eb5d46adc6d3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:04.677850 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 7/15] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:14:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 18:14:04.683967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c756e213-496d-43b0-8295-6ad1b7e92306 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:04.763677 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0012c16-3274-4d75-87f1-eb5d46adc6d3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:04.786249 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c756e213-496d-43b0-8295-6ad1b7e92306 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:04.798019 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f97041-9126-40d7-9ae0-3d4d29d7a1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:04.798700 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f97041-9126-40d7-9ae0-3d4d29d7a1cf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:04.798985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f97041-9126-40d7-9ae0-3d4d29d7a1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:04.799268 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f97041-9126-40d7-9ae0-3d4d29d7a1cf None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:04.800034 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f97041-9126-40d7-9ae0-3d4d29d7a1cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:04.800340 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f97041-9126-40d7-9ae0-3d4d29d7a1cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:04.801820 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f97041-9126-40d7-9ae0-3d4d29d7a1cf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:04.810103 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:04.817195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 9/16] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:04.823622 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b918d-06de-4677-b02f-7d8aecb0a9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:04.823979 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f97041-9126-40d7-9ae0-3d4d29d7a1cf None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:04.824469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b918d-06de-4677-b02f-7d8aecb0a9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:04.824757 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b918d-06de-4677-b02f-7d8aecb0a9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:04.825035 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b918d-06de-4677-b02f-7d8aecb0a9b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:04.836194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62b918d-06de-4677-b02f-7d8aecb0a9b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:04.838592 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62b918d-06de-4677-b02f-7d8aecb0a9b6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:04.852763 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:04.860670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b62b918d-06de-4677-b02f-7d8aecb0a9b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:14:04.860670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:04.860670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:04.860670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:04.860670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:04.860670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:04.860670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:04.860670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:04.860670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:04.860670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:04.860670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:04.860670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:04.860670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:14:04.860670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:14:04.860670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:14:04.861927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:14:04.861927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:04.861927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:04.861927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:04.861927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:04.861927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:04.861927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:04.861927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:04.861927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:04.861927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:04.861927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:04.861927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:04.861927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:04.861927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:04.861927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:04.861927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:04.861927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:04.862730 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:14:04.862730 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:14:04.862730 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:14:04.862730 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:14:04.862730 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 8/17] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:14:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:14:04.868270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b918d-06de-4677-b02f-7d8aecb0a9b6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:04.934841 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f97041-9126-40d7-9ae0-3d4d29d7a1cf None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:04.969491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5451a-2426-46aa-b44b-75ae33a59552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:04.970388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5451a-2426-46aa-b44b-75ae33a59552 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:04.970671 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5451a-2426-46aa-b44b-75ae33a59552 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:04.970946 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5451a-2426-46aa-b44b-75ae33a59552 None admin] PATH_INFO: `/v3/users/de745ad7f2ed48ffbb043529099e53f0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:04.971055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b918d-06de-4677-b02f-7d8aecb0a9b6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:04.972503 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5451a-2426-46aa-b44b-75ae33a59552 None admin] RBAC: Authorizing `identity:get_user(user_id=de745ad7f2ed48ffbb043529099e53f0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:04.974000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5451a-2426-46aa-b44b-75ae33a59552 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:04.975715 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 10/18] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:04 2022] GET /identity/v3/users/de745ad7f2ed48ffbb043529099e53f0 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:04.981560 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5451a-2426-46aa-b44b-75ae33a59552 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:05.006063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876915d-247e-45da-9d9a-5ebe9e9a1fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:05.006777 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876915d-247e-45da-9d9a-5ebe9e9a1fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:05.007052 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876915d-247e-45da-9d9a-5ebe9e9a1fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:05.007413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876915d-247e-45da-9d9a-5ebe9e9a1fb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:05.008116 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876915d-247e-45da-9d9a-5ebe9e9a1fb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:05.008460 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876915d-247e-45da-9d9a-5ebe9e9a1fb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:05.010457 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876915d-247e-45da-9d9a-5ebe9e9a1fb2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:05.030747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 9/19] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:14:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:05.092558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:05.099974 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5451a-2426-46aa-b44b-75ae33a59552 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:05.136678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627da9b4-bb63-4eb4-ae09-4296ea50beb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:05.137481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627da9b4-bb63-4eb4-ae09-4296ea50beb8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:05.137807 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627da9b4-bb63-4eb4-ae09-4296ea50beb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:05.138088 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627da9b4-bb63-4eb4-ae09-4296ea50beb8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:05.139331 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627da9b4-bb63-4eb4-ae09-4296ea50beb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:05.140834 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627da9b4-bb63-4eb4-ae09-4296ea50beb8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:05.142812 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 11/20] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:14:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:05.148556 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876915d-247e-45da-9d9a-5ebe9e9a1fb2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:05.185225 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:05.253989 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876915d-247e-45da-9d9a-5ebe9e9a1fb2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:05.292484 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5a2c9-a882-44e4-8049-33b329b2ce6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:05.293153 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5a2c9-a882-44e4-8049-33b329b2ce6f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:05.293444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5a2c9-a882-44e4-8049-33b329b2ce6f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:05.293759 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5a2c9-a882-44e4-8049-33b329b2ce6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:05.294860 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff5a2c9-a882-44e4-8049-33b329b2ce6f None admin] RBAC: Adding query filter params (role.id=0ef3687af20a443281e3cfdceca3597d, scope.domain.id=default, user.id=de745ad7f2ed48ffbb043529099e53f0) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:05.295180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff5a2c9-a882-44e4-8049-33b329b2ce6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:05.296641 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff5a2c9-a882-44e4-8049-33b329b2ce6f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:05.303676 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 10/21] 10.222.0.33 () {56 vars in 1322 bytes} [Thu Jun 30 18:14:05 2022] GET /identity/v3/role_assignments?user.id=de745ad7f2ed48ffbb043529099e53f0&scope.domain.id=default&role.id=0ef3687af20a443281e3cfdceca3597d => generated 233 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:07.661982 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cef440-aa10-483c-a8af-fa3d08d66e21 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:07.663009 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cef440-aa10-483c-a8af-fa3d08d66e21 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:07.663512 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cef440-aa10-483c-a8af-fa3d08d66e21 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:07.665109 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 12/22] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:07.671788 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decee52-56f5-4ea9-9071-aa929dd76675 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:07.672696 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decee52-56f5-4ea9-9071-aa929dd76675 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:07.673174 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decee52-56f5-4ea9-9071-aa929dd76675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:07.695142 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6decee52-56f5-4ea9-9071-aa929dd76675 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:07.697433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6decee52-56f5-4ea9-9071-aa929dd76675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:07.798617 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 11/23] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:07.839972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cef440-aa10-483c-a8af-fa3d08d66e21 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:07.846755 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d856bd-151d-46e1-ab6b-7a4e8446d9d9 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:07.847328 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d856bd-151d-46e1-ab6b-7a4e8446d9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:07.847827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d856bd-151d-46e1-ab6b-7a4e8446d9d9 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:07.849403 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 12/24] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:14:07.867140 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06faff67-5185-41d0-9148-83ed04fd1a5c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:07.867782 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06faff67-5185-41d0-9148-83ed04fd1a5c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:07.868262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06faff67-5185-41d0-9148-83ed04fd1a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:07.904356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-06faff67-5185-41d0-9148-83ed04fd1a5c None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:07.907116 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-06faff67-5185-41d0-9148-83ed04fd1a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:07.966147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:08.049562 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 13/25] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 18:14:08.051031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d75a1-b2a4-469a-a7b9-9ab1b14b2773 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:08.051326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d75a1-b2a4-469a-a7b9-9ab1b14b2773 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:08.051863 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d75a1-b2a4-469a-a7b9-9ab1b14b2773 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:08.052600 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cef440-aa10-483c-a8af-fa3d08d66e21 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:08.053281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 14/26] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:08.071117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56445e-6061-4d15-a552-964acb2411a3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:08.072275 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56445e-6061-4d15-a552-964acb2411a3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:08.072852 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56445e-6061-4d15-a552-964acb2411a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:08.096811 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3b56445e-6061-4d15-a552-964acb2411a3 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:08.098410 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3b56445e-6061-4d15-a552-964acb2411a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:08.143653 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d02301-55e2-420c-9fce-4b395bb07591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:08.144335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d02301-55e2-420c-9fce-4b395bb07591 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:08.144626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d02301-55e2-420c-9fce-4b395bb07591 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:08.144913 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d02301-55e2-420c-9fce-4b395bb07591 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:08.145704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d02301-55e2-420c-9fce-4b395bb07591 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:08.147243 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d02301-55e2-420c-9fce-4b395bb07591 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:08.177139 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 15/27] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:08.178347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56445e-6061-4d15-a552-964acb2411a3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:08.298381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:08.302756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56445e-6061-4d15-a552-964acb2411a3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:08.314775 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 13/28] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Jun 30 18:14:07 2022] POST /identity/v3/roles => generated 219 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:14:08.316207 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d02301-55e2-420c-9fce-4b395bb07591 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:08.343767 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e07fab0-5ae3-4936-ae80-d1b030e6d716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:08.344681 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07fab0-5ae3-4936-ae80-d1b030e6d716 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:08.345169 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07fab0-5ae3-4936-ae80-d1b030e6d716 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:08.345634 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07fab0-5ae3-4936-ae80-d1b030e6d716 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:08.352806 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:08.360712 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e07fab0-5ae3-4936-ae80-d1b030e6d716 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:08.370398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-6e07fab0-5ae3-4936-ae80-d1b030e6d716 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:14:08.370398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:08.370398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:08.370398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:08.370398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:08.370398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:08.370398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:08.370398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:08.370398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:08.370398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:08.370398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:08.370398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:08.370398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:08.370398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:08.370398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:08.370398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:08.371330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:08.371330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:08.371330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:08.371330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:08.371330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:08.371330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:08.371330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:08.371330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:08.371330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:08.371330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:08.371330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:08.371330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:08.371330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:08.371330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:08.371330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:08.371330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:08.372080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:08.372080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:08.372080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:08.372080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:08.372080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:08.372080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:08.372080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:08.372080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:08.372080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:08.372080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:08.372080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:14:08.372080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:14:08.374769 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 16/29] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:14:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:08.376389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ecbd8-0cb2-4547-8b5d-59f9ac96ba89 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:08.376889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ecbd8-0cb2-4547-8b5d-59f9ac96ba89 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:08.377379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ecbd8-0cb2-4547-8b5d-59f9ac96ba89 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:08.379046 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 17/30] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:08.384976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f274d4c-8a16-4f96-b64c-da0ad4f21f62 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:08.385553 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f274d4c-8a16-4f96-b64c-da0ad4f21f62 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:08.388357 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f274d4c-8a16-4f96-b64c-da0ad4f21f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:08.421908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2f274d4c-8a16-4f96-b64c-da0ad4f21f62 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:08.423201 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2f274d4c-8a16-4f96-b64c-da0ad4f21f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:08.523984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d02301-55e2-420c-9fce-4b395bb07591 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:08.541240 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 18/31] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:08.542607 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f274d4c-8a16-4f96-b64c-da0ad4f21f62 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:08.602914 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db401611-b273-4ddb-9d83-11ed222bd659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:08.604012 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db401611-b273-4ddb-9d83-11ed222bd659 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:08.604299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db401611-b273-4ddb-9d83-11ed222bd659 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:08.604581 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db401611-b273-4ddb-9d83-11ed222bd659 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:08.627637 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db401611-b273-4ddb-9d83-11ed222bd659 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:08.629313 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db401611-b273-4ddb-9d83-11ed222bd659 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:08.656702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-db401611-b273-4ddb-9d83-11ed222bd659 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:14:08.656702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:08.656702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:08.656702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:08.656702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:08.656702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:08.656702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:08.656702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:08.656702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:08.656702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:08.656702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:08.656702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:08.656702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:14:08.656702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:14:08.656702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:14:08.657777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:14:08.657777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:08.657777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:08.657777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:08.657777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:08.657777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:08.657777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:08.657777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:08.657777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:08.657777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:08.657777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:08.657777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:08.657777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:08.657777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:08.657777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:08.657777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:08.657777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:08.658678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:14:08.658678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:14:08.658678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:14:08.658678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:14:08.659196 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:08.661396 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 14/32] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:14:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:08.662503 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db401611-b273-4ddb-9d83-11ed222bd659 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:08.701907 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f274d4c-8a16-4f96-b64c-da0ad4f21f62 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:08.738756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c6458-71ab-47d9-bb8e-6454dc34562f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:08.739484 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c6458-71ab-47d9-bb8e-6454dc34562f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:08.739764 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c6458-71ab-47d9-bb8e-6454dc34562f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:08.740200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c6458-71ab-47d9-bb8e-6454dc34562f None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:08.741106 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c6458-71ab-47d9-bb8e-6454dc34562f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:08.741554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c6458-71ab-47d9-bb8e-6454dc34562f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:08.743117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c6458-71ab-47d9-bb8e-6454dc34562f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:08.756410 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 19/33] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:08.757452 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c6458-71ab-47d9-bb8e-6454dc34562f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:08.771543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:08.800516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db401611-b273-4ddb-9d83-11ed222bd659 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:08.837940 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c054f-cc42-4ded-8be8-5ee03551db6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:08.839104 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c054f-cc42-4ded-8be8-5ee03551db6d None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:08.839555 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c054f-cc42-4ded-8be8-5ee03551db6d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:08.839978 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c054f-cc42-4ded-8be8-5ee03551db6d None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:08.840873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c054f-cc42-4ded-8be8-5ee03551db6d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:08.842633 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c054f-cc42-4ded-8be8-5ee03551db6d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:08.865328 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 15/34] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Jun 30 18:14:08 2022] POST /identity/v3/roles => generated 225 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:14:08.867140 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c054f-cc42-4ded-8be8-5ee03551db6d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:08.888735 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c6458-71ab-47d9-bb8e-6454dc34562f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:08.956260 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557ee1d-4ada-403f-a903-806d896a0658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:08.956942 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557ee1d-4ada-403f-a903-806d896a0658 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:08.957228 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557ee1d-4ada-403f-a903-806d896a0658 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:08.957549 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557ee1d-4ada-403f-a903-806d896a0658 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:08.960634 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557ee1d-4ada-403f-a903-806d896a0658 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:08.960949 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557ee1d-4ada-403f-a903-806d896a0658 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:08.964458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557ee1d-4ada-403f-a903-806d896a0658 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:08.980109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:08.988097 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 20/35] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:14:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:08.995206 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557ee1d-4ada-403f-a903-806d896a0658 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:09.100523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c054f-cc42-4ded-8be8-5ee03551db6d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:09.140721 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557ee1d-4ada-403f-a903-806d896a0658 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:09.144230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857ea34-7733-492e-8a01-ed3c3727d909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:09.145123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857ea34-7733-492e-8a01-ed3c3727d909 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:09.145412 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857ea34-7733-492e-8a01-ed3c3727d909 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:09.145735 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857ea34-7733-492e-8a01-ed3c3727d909 None admin] PATH_INFO: `/v3/users/de745ad7f2ed48ffbb043529099e53f0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:09.147128 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857ea34-7733-492e-8a01-ed3c3727d909 None admin] RBAC: Authorizing `identity:get_user(user_id=de745ad7f2ed48ffbb043529099e53f0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:09.148867 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857ea34-7733-492e-8a01-ed3c3727d909 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:09.153632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 16/36] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:08 2022] GET /identity/v3/users/de745ad7f2ed48ffbb043529099e53f0 => generated 247 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:09.153632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a0b42-cac0-4d06-a867-f5b06eedd3c7 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:09.153632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a0b42-cac0-4d06-a867-f5b06eedd3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:09.153632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a0b42-cac0-4d06-a867-f5b06eedd3c7 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:09.154897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 17/37] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:09.160792 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9bdd9-9851-485d-b30d-884b9d053af5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:09.161161 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9bdd9-9851-485d-b30d-884b9d053af5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:09.161536 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9bdd9-9851-485d-b30d-884b9d053af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:09.166663 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:09.182316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7ec9bdd9-9851-485d-b30d-884b9d053af5 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:09.182996 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2caea-d1ff-4f84-8ac4-33776a34b659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:09.183667 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2caea-d1ff-4f84-8ac4-33776a34b659 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:09.183901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7ec9bdd9-9851-485d-b30d-884b9d053af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:09.183991 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2caea-d1ff-4f84-8ac4-33776a34b659 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:09.184224 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2caea-d1ff-4f84-8ac4-33776a34b659 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:09.184963 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2caea-d1ff-4f84-8ac4-33776a34b659 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:09.186542 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2caea-d1ff-4f84-8ac4-33776a34b659 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:09.252754 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 18/38] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:09.253655 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9bdd9-9851-485d-b30d-884b9d053af5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:09.358275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9bdd9-9851-485d-b30d-884b9d053af5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:09.392006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:09.398417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663d64d9-424d-4f8b-899c-f3d8aa46da8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:09.399268 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d64d9-424d-4f8b-899c-f3d8aa46da8f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:09.399575 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d64d9-424d-4f8b-899c-f3d8aa46da8f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:09.399920 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d64d9-424d-4f8b-899c-f3d8aa46da8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:09.401228 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d64d9-424d-4f8b-899c-f3d8aa46da8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:09.402783 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d64d9-424d-4f8b-899c-f3d8aa46da8f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:09.404524 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 19/39] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:14:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:09.405857 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9b922-3c13-440f-91c5-7297f047a19f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:09.406153 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9b922-3c13-440f-91c5-7297f047a19f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:09.406434 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9b922-3c13-440f-91c5-7297f047a19f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:09.409633 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 20/40] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:09.409633 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9b922-3c13-440f-91c5-7297f047a19f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:09.432879 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 21/41] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 30 18:14:08 2022] POST /identity/v3/projects => generated 301 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:14:09.434066 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2caea-d1ff-4f84-8ac4-33776a34b659 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:09.503151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:09.550162 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9b922-3c13-440f-91c5-7297f047a19f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:09.576473 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2caea-d1ff-4f84-8ac4-33776a34b659 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:09.597799 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1cd5ee-ebc3-4dff-88f5-e16d50dfa53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:09.599379 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1cd5ee-ebc3-4dff-88f5-e16d50dfa53a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:09.599379 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1cd5ee-ebc3-4dff-88f5-e16d50dfa53a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:09.599379 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1cd5ee-ebc3-4dff-88f5-e16d50dfa53a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:09.600441 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1cd5ee-ebc3-4dff-88f5-e16d50dfa53a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:09.601989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1cd5ee-ebc3-4dff-88f5-e16d50dfa53a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:09.604684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 21/42] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:14:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:09.605907 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4beb09-6fbe-4383-9ee6-485b2a090fa4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:09.606372 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4beb09-6fbe-4383-9ee6-485b2a090fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:09.606677 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4beb09-6fbe-4383-9ee6-485b2a090fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:09.626351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5a4beb09-6fbe-4383-9ee6-485b2a090fa4 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:09.632529 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5a4beb09-6fbe-4383-9ee6-485b2a090fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:09.638971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b1691-2b8b-41bc-bfa3-155fe3543ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:09.639872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b1691-2b8b-41bc-bfa3-155fe3543ac4 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:09.640163 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b1691-2b8b-41bc-bfa3-155fe3543ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:09.640445 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b1691-2b8b-41bc-bfa3-155fe3543ac4 None admin] PATH_INFO: `/v3/domains/default/users/de745ad7f2ed48ffbb043529099e53f0/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:09.642813 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b1691-2b8b-41bc-bfa3-155fe3543ac4 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=de745ad7f2ed48ffbb043529099e53f0, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:09.644364 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b1691-2b8b-41bc-bfa3-155fe3543ac4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:09.665952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 22/43] 10.222.0.33 () {58 vars in 1361 bytes} [Thu Jun 30 18:14:09 2022] PUT /identity/v3/domains/default/users/de745ad7f2ed48ffbb043529099e53f0/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:14:09.666792 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b1691-2b8b-41bc-bfa3-155fe3543ac4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:09.717861 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:09.725974 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 22/44] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:09.748283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4beb09-6fbe-4383-9ee6-485b2a090fa4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:09.823846 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b1691-2b8b-41bc-bfa3-155fe3543ac4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:09.847528 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:09.871039 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae9509-72ab-488e-83e9-1d693f61a204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:09.871715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae9509-72ab-488e-83e9-1d693f61a204 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:09.872007 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae9509-72ab-488e-83e9-1d693f61a204 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:09.872295 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae9509-72ab-488e-83e9-1d693f61a204 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:09.873035 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae9509-72ab-488e-83e9-1d693f61a204 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:09.874546 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4beb09-6fbe-4383-9ee6-485b2a090fa4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:09.874655 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae9509-72ab-488e-83e9-1d693f61a204 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:09.915830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 23/45] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 30 18:14:09 2022] POST /identity/v3/projects => generated 312 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:14:09.925188 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b4a032-a6ee-46a9-a904-9336f65447fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:09.926383 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b4a032-a6ee-46a9-a904-9336f65447fd None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:09.926695 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b4a032-a6ee-46a9-a904-9336f65447fd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:09.926981 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b4a032-a6ee-46a9-a904-9336f65447fd None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:09.928259 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b4a032-a6ee-46a9-a904-9336f65447fd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:09.929903 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b4a032-a6ee-46a9-a904-9336f65447fd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:09.950617 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 23/46] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Jun 30 18:14:09 2022] POST /identity/v3/roles => generated 223 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:14:11.881419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7164283-d096-4d81-aad3-a22e7dd4b057 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:11.882231 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7164283-d096-4d81-aad3-a22e7dd4b057 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:11.882231 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7164283-d096-4d81-aad3-a22e7dd4b057 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:11.883609 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 24/47] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:11.890206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e23ffd-48bc-4edb-b84c-cd2b01cf396c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:11.891573 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e23ffd-48bc-4edb-b84c-cd2b01cf396c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:11.891573 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e23ffd-48bc-4edb-b84c-cd2b01cf396c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:11.908361 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e4e23ffd-48bc-4edb-b84c-cd2b01cf396c None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:11.910080 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e4e23ffd-48bc-4edb-b84c-cd2b01cf396c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:11.977672 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:12.001202 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 24/48] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:12.034603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7164283-d096-4d81-aad3-a22e7dd4b057 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:12.092160 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d82ce8-b01d-4be9-b1d8-c7f608604163 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:12.092447 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d82ce8-b01d-4be9-b1d8-c7f608604163 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:12.092740 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d82ce8-b01d-4be9-b1d8-c7f608604163 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:12.095106 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 25/49] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:14:12.100932 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40160142-1922-4b25-a79d-7cb47f2a3ee8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:12.101223 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40160142-1922-4b25-a79d-7cb47f2a3ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:12.101510 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40160142-1922-4b25-a79d-7cb47f2a3ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:12.126925 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-40160142-1922-4b25-a79d-7cb47f2a3ee8 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:12.128261 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-40160142-1922-4b25-a79d-7cb47f2a3ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:12.188034 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7164283-d096-4d81-aad3-a22e7dd4b057 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:12.217558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 26/50] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:12.223603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a5515-1b15-4620-9c74-c6d2f998d3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:12.224404 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a5515-1b15-4620-9c74-c6d2f998d3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:12.224694 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a5515-1b15-4620-9c74-c6d2f998d3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:12.224974 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a5515-1b15-4620-9c74-c6d2f998d3f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:12.238048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a5515-1b15-4620-9c74-c6d2f998d3f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:12.241272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40160142-1922-4b25-a79d-7cb47f2a3ee8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:12.252960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c15a5515-1b15-4620-9c74-c6d2f998d3f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:14:12.252960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:12.252960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:12.252960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:12.252960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:12.252960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:12.252960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:12.252960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:12.252960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:12.252960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:12.252960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:12.252960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:12.252960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:12.252960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:12.252960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:12.252960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:12.253921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:12.253921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:12.253921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:12.253921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:12.253921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:12.253921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:12.253921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:12.253921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:12.253921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:12.253921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:12.253921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:12.253921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:12.253921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:12.253921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:12.253921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:12.253921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:12.254654 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:12.254654 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:12.254654 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:12.254654 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:12.254654 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:12.254654 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:12.254654 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:12.254654 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:12.254654 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:12.254654 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:12.254654 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:14:12.254654 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:14:12.254654 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 25/51] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:14:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:12.262139 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a5515-1b15-4620-9c74-c6d2f998d3f5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:12.289553 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:12.366035 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40160142-1922-4b25-a79d-7cb47f2a3ee8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:12.380688 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a5515-1b15-4620-9c74-c6d2f998d3f5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:12.424545 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b491f16-811d-411f-bf68-7be0bdb03853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:12.425239 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b491f16-811d-411f-bf68-7be0bdb03853 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:12.425523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b491f16-811d-411f-bf68-7be0bdb03853 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:12.425836 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b491f16-811d-411f-bf68-7be0bdb03853 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:12.426632 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b491f16-811d-411f-bf68-7be0bdb03853 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:12.426945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b491f16-811d-411f-bf68-7be0bdb03853 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:12.428422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b491f16-811d-411f-bf68-7be0bdb03853 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:12.442699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 26/52] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:12.449611 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b491f16-811d-411f-bf68-7be0bdb03853 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:12.497125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:12.517720 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d2742-e8d2-4255-bb87-7d330e251b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:12.518536 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d2742-e8d2-4255-bb87-7d330e251b11 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:12.518825 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d2742-e8d2-4255-bb87-7d330e251b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:12.519102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d2742-e8d2-4255-bb87-7d330e251b11 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:12.520297 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d2742-e8d2-4255-bb87-7d330e251b11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:12.521802 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d2742-e8d2-4255-bb87-7d330e251b11 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:12.523457 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 27/53] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:14:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:12.532219 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d2742-e8d2-4255-bb87-7d330e251b11 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:12.549563 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:12.573044 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b491f16-811d-411f-bf68-7be0bdb03853 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:12.610019 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6512c96-e204-4b97-85ad-3f7206a0b474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:12.610905 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6512c96-e204-4b97-85ad-3f7206a0b474 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:12.611197 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6512c96-e204-4b97-85ad-3f7206a0b474 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:12.611484 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6512c96-e204-4b97-85ad-3f7206a0b474 None admin] PATH_INFO: `/v3/users/de745ad7f2ed48ffbb043529099e53f0` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:12.612733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6512c96-e204-4b97-85ad-3f7206a0b474 None admin] RBAC: Authorizing `identity:get_user(user_id=de745ad7f2ed48ffbb043529099e53f0)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:12.614288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6512c96-e204-4b97-85ad-3f7206a0b474 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:12.615966 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 27/54] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:12 2022] GET /identity/v3/users/de745ad7f2ed48ffbb043529099e53f0 => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:12.622307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6512c96-e204-4b97-85ad-3f7206a0b474 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:12.646283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d2742-e8d2-4255-bb87-7d330e251b11 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:12.685852 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85916c2-fae3-40c1-b661-f0a1d7f6e93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:12.686653 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85916c2-fae3-40c1-b661-f0a1d7f6e93b None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:12.687057 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85916c2-fae3-40c1-b661-f0a1d7f6e93b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:12.687445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85916c2-fae3-40c1-b661-f0a1d7f6e93b None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:12.688261 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85916c2-fae3-40c1-b661-f0a1d7f6e93b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:12.689846 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85916c2-fae3-40c1-b661-f0a1d7f6e93b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:12.724744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 28/55] 10.222.0.33 () {60 vars in 1140 bytes} [Thu Jun 30 18:14:12 2022] POST /identity/v3/projects => generated 298 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:14:12.737349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6512c96-e204-4b97-85ad-3f7206a0b474 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:12.779384 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f2919e-bf63-40fe-bb01-43a4b3f8d493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:12.780209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2919e-bf63-40fe-bb01-43a4b3f8d493 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:12.780499 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2919e-bf63-40fe-bb01-43a4b3f8d493 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:12.780784 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2919e-bf63-40fe-bb01-43a4b3f8d493 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:12.782263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f2919e-bf63-40fe-bb01-43a4b3f8d493 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:12.783868 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f2919e-bf63-40fe-bb01-43a4b3f8d493 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:12.785755 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 28/56] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:14:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:12.792045 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85916c2-fae3-40c1-b661-f0a1d7f6e93b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:12.838695 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:12.916845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85916c2-fae3-40c1-b661-f0a1d7f6e93b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:12.962510 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23914bcc-5e90-4009-b00d-36dfeb23c48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:12.963193 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23914bcc-5e90-4009-b00d-36dfeb23c48f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:12.963483 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23914bcc-5e90-4009-b00d-36dfeb23c48f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:12.963769 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23914bcc-5e90-4009-b00d-36dfeb23c48f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:12.964626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23914bcc-5e90-4009-b00d-36dfeb23c48f None admin] RBAC: Adding query filter params (role.id=0ef3687af20a443281e3cfdceca3597d, scope.domain.id=default, user.id=de745ad7f2ed48ffbb043529099e53f0) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:12.964939 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23914bcc-5e90-4009-b00d-36dfeb23c48f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:12.966463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23914bcc-5e90-4009-b00d-36dfeb23c48f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:12.975988 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 29/57] 10.222.0.33 () {56 vars in 1322 bytes} [Thu Jun 30 18:14:12 2022] GET /identity/v3/role_assignments?user.id=de745ad7f2ed48ffbb043529099e53f0&scope.domain.id=default&role.id=0ef3687af20a443281e3cfdceca3597d => generated 530 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:14.849054 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3338a9a-1dda-419b-b411-6af2ca6662a9 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:14.849644 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3338a9a-1dda-419b-b411-6af2ca6662a9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:14.849644 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3338a9a-1dda-419b-b411-6af2ca6662a9 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:14.851041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 29/58] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:14.856580 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75551f78-b327-4d33-bae9-950fc22d457c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:14.857119 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75551f78-b327-4d33-bae9-950fc22d457c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:14.857260 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75551f78-b327-4d33-bae9-950fc22d457c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:14.874697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-75551f78-b327-4d33-bae9-950fc22d457c None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:14.875928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-75551f78-b327-4d33-bae9-950fc22d457c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:14.947947 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 30/59] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:14.972259 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3338a9a-1dda-419b-b411-6af2ca6662a9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:14.997950 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:15.084831 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3338a9a-1dda-419b-b411-6af2ca6662a9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:15.121229 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09d35f-c980-4329-9574-f75a7c0b32d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:15.122004 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09d35f-c980-4329-9574-f75a7c0b32d5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:15.122323 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09d35f-c980-4329-9574-f75a7c0b32d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:15.122611 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09d35f-c980-4329-9574-f75a7c0b32d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:15.123823 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09d35f-c980-4329-9574-f75a7c0b32d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:15.125286 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09d35f-c980-4329-9574-f75a7c0b32d5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:15.128186 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 30/60] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:14:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:15.134618 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75551f78-b327-4d33-bae9-950fc22d457c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:15.166140 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:15.235827 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75551f78-b327-4d33-bae9-950fc22d457c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:15.271083 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba82c6f-8376-4cb2-95bb-94c126557515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:15.271824 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba82c6f-8376-4cb2-95bb-94c126557515 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:15.272115 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba82c6f-8376-4cb2-95bb-94c126557515 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:15.272400 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba82c6f-8376-4cb2-95bb-94c126557515 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:15.273112 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba82c6f-8376-4cb2-95bb-94c126557515 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:15.275028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba82c6f-8376-4cb2-95bb-94c126557515 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:15.302074 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 31/61] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 30 18:14:15 2022] POST /identity/v3/users => generated 275 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:14:18.686920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1101f-be33-4a3a-9240-f4081ecca109 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:18.687485 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1101f-be33-4a3a-9240-f4081ecca109 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:18.687485 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1101f-be33-4a3a-9240-f4081ecca109 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:18.688872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 31/62] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:18.702970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff28ec-5737-487a-abe1-8afc3b5e047e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:18.703500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff28ec-5737-487a-abe1-8afc3b5e047e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:18.703583 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff28ec-5737-487a-abe1-8afc3b5e047e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:18.728112 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-13ff28ec-5737-487a-abe1-8afc3b5e047e None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:18.729313 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-13ff28ec-5737-487a-abe1-8afc3b5e047e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:18.802594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929eb26e-c4ae-4ffd-abba-669a62014b52 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:18.802922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929eb26e-c4ae-4ffd-abba-669a62014b52 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:18.803153 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929eb26e-c4ae-4ffd-abba-669a62014b52 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:18.804552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 32/63] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:14:18.811212 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576dd74d-550b-413e-8ed5-f1d32fadde71 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:18.811832 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576dd74d-550b-413e-8ed5-f1d32fadde71 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:18.812317 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576dd74d-550b-413e-8ed5-f1d32fadde71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:18.836935 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-576dd74d-550b-413e-8ed5-f1d32fadde71 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:18.838255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 32/64] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:18.839520 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-576dd74d-550b-413e-8ed5-f1d32fadde71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:18.840252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f7c90-cf80-4cb7-95a8-8add597867f0 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:18.840548 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f7c90-cf80-4cb7-95a8-8add597867f0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:18.840862 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f7c90-cf80-4cb7-95a8-8add597867f0 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:18.843041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 33/65] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:18.853603 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da838f-e8be-4964-9ca3-4dac543cabce None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:18.853603 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da838f-e8be-4964-9ca3-4dac543cabce None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:18.853603 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da838f-e8be-4964-9ca3-4dac543cabce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:18.880123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f9da838f-e8be-4964-9ca3-4dac543cabce None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:18.882359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f9da838f-e8be-4964-9ca3-4dac543cabce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:18.901628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:18.936303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 33/66] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 18:14:18.937545 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576dd74d-550b-413e-8ed5-f1d32fadde71 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:18.981220 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:18.989165 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 34/67] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:18.990297 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da838f-e8be-4964-9ca3-4dac543cabce None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:19.112473 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576dd74d-550b-413e-8ed5-f1d32fadde71 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:19.153985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da838f-e8be-4964-9ca3-4dac543cabce None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:19.154201 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec11b2e-52b9-4870-93c9-f25c42dd6e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:19.154859 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec11b2e-52b9-4870-93c9-f25c42dd6e26 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:19.155153 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec11b2e-52b9-4870-93c9-f25c42dd6e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:19.155440 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec11b2e-52b9-4870-93c9-f25c42dd6e26 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:19.175433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec11b2e-52b9-4870-93c9-f25c42dd6e26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:19.186560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-9ec11b2e-52b9-4870-93c9-f25c42dd6e26 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:14:19.186560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:19.186560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:19.186560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:19.186560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:19.186560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:19.186560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:19.186560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:19.186560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:19.186560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:19.186560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:19.186560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:19.186560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:19.186560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:19.186560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:19.186560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:19.187522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:19.187522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:19.187522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:19.187522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:19.187522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:19.187522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:19.187522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:19.187522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:19.187522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:19.187522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:19.187522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:19.187522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:19.187522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:19.187522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:19.187522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:19.187522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:19.188356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:19.188356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:19.188356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:19.188356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:19.188356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:19.188356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:19.188356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:19.188356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:19.188356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:19.188356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:19.188356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:14:19.188356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:14:19.188356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 34/68] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:19.189496 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec11b2e-52b9-4870-93c9-f25c42dd6e26 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:19.207626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037c588-62cd-4e74-a326-ec786258d094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:19.208817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037c588-62cd-4e74-a326-ec786258d094 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:19.209326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037c588-62cd-4e74-a326-ec786258d094 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:19.209838 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037c588-62cd-4e74-a326-ec786258d094 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:19.223557 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7037c588-62cd-4e74-a326-ec786258d094 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:19.232654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-7037c588-62cd-4e74-a326-ec786258d094 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:14:19.232654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:19.232654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:19.232654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:19.232654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:19.232654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:19.232654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:19.232654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:19.232654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:19.232654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:19.232654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:19.232654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:19.232654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:19.232654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:19.232654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:19.232654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:19.233539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:19.233539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:19.233539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:19.233539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:19.233539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:19.233539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:19.233539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:19.233539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:19.233539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:19.233539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:19.233539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:19.233539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:19.233539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:19.233539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:19.233539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:19.233539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:19.234323 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:19.234323 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:19.234323 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:19.234323 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:19.234323 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:19.234323 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:19.234323 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:19.234323 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:19.234323 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:19.234323 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:19.234323 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:14:19.234323 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:14:19.237912 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 35/69] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 30 18:14:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:19.238755 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037c588-62cd-4e74-a326-ec786258d094 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:19.253281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:19.293897 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:19.316247 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec11b2e-52b9-4870-93c9-f25c42dd6e26 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:19.351635 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b396d5-abe7-4bc5-893e-46287bcdca6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:19.353035 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b396d5-abe7-4bc5-893e-46287bcdca6e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:19.353318 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b396d5-abe7-4bc5-893e-46287bcdca6e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:19.353623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b396d5-abe7-4bc5-893e-46287bcdca6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:19.354310 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037c588-62cd-4e74-a326-ec786258d094 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:19.367926 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b396d5-abe7-4bc5-893e-46287bcdca6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:19.375754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c1b396d5-abe7-4bc5-893e-46287bcdca6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:14:19.375754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:19.375754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:19.375754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:19.375754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:19.375754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:19.375754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:19.375754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:19.375754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:19.375754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:19.375754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:19.375754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:19.375754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:19.375754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:19.375754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:19.375754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:19.376710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:19.376710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:19.376710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:19.376710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:19.376710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:19.376710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:19.376710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:19.376710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:19.376710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:19.376710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:19.376710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:19.376710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:19.376710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:19.376710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:19.376710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:19.376710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:19.377464 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:19.377464 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:19.377464 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:19.377464 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:19.377464 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:19.377464 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:19.377464 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:19.377464 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:19.377464 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:19.377464 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:19.377464 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:14:19.377464 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:14:19.377464 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 35/70] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:14:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:19.378644 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085c3bc-2f16-4995-a6b7-50731a01d93b None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:19.378746 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085c3bc-2f16-4995-a6b7-50731a01d93b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:19.379002 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085c3bc-2f16-4995-a6b7-50731a01d93b None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:19.380467 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 36/71] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:19.382688 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085c3bc-2f16-4995-a6b7-50731a01d93b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:19.404218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0102a9c-facc-4fae-b1fe-01ac9778b7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:19.405383 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0102a9c-facc-4fae-b1fe-01ac9778b7ec None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:19.405910 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0102a9c-facc-4fae-b1fe-01ac9778b7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:19.406437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0102a9c-facc-4fae-b1fe-01ac9778b7ec None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:19.407916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0102a9c-facc-4fae-b1fe-01ac9778b7ec None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:19.408507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0102a9c-facc-4fae-b1fe-01ac9778b7ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:19.411348 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0102a9c-facc-4fae-b1fe-01ac9778b7ec None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:19.423768 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 36/72] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:14:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:19.425380 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda75b5-770c-4e5a-ac4c-1089a7764d0b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:19.425893 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda75b5-770c-4e5a-ac4c-1089a7764d0b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:19.426355 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda75b5-770c-4e5a-ac4c-1089a7764d0b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:19.428076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 37/73] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:19.429186 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda75b5-770c-4e5a-ac4c-1089a7764d0b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:19.502039 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085c3bc-2f16-4995-a6b7-50731a01d93b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:19.514061 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:19.548670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c7fde-7a7b-4f98-b5c7-300519508d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:19.549352 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c7fde-7a7b-4f98-b5c7-300519508d91 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:19.549674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c7fde-7a7b-4f98-b5c7-300519508d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:19.549963 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c7fde-7a7b-4f98-b5c7-300519508d91 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:19.550213 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda75b5-770c-4e5a-ac4c-1089a7764d0b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:19.550737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c7fde-7a7b-4f98-b5c7-300519508d91 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:19.551056 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c7fde-7a7b-4f98-b5c7-300519508d91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:19.552501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c7fde-7a7b-4f98-b5c7-300519508d91 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:19.563621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 37/74] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:14:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:19.565396 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fb5db-85fd-4b9f-91df-ffb0b9b376a1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:19.565736 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fb5db-85fd-4b9f-91df-ffb0b9b376a1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:19.566013 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fb5db-85fd-4b9f-91df-ffb0b9b376a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:19.571948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:19.587758 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e25fb5db-85fd-4b9f-91df-ffb0b9b376a1 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:19.588990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e25fb5db-85fd-4b9f-91df-ffb0b9b376a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:19.589962 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c282a27-e052-49a2-8c2f-f8e80d841fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:19.590649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c282a27-e052-49a2-8c2f-f8e80d841fea None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:19.590944 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c282a27-e052-49a2-8c2f-f8e80d841fea None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:19.591233 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c282a27-e052-49a2-8c2f-f8e80d841fea None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:19.592002 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c282a27-e052-49a2-8c2f-f8e80d841fea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:19.592326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c282a27-e052-49a2-8c2f-f8e80d841fea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:19.593779 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c282a27-e052-49a2-8c2f-f8e80d841fea None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:19.603148 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 38/75] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:19.609292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c282a27-e052-49a2-8c2f-f8e80d841fea None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:19.687512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 38/76] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:19.688582 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc9111-b289-439b-b62e-644603aafa00 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:19.688871 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc9111-b289-439b-b62e-644603aafa00 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:19.689152 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc9111-b289-439b-b62e-644603aafa00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:19.706951 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4cdc9111-b289-439b-b62e-644603aafa00 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:19.708157 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4cdc9111-b289-439b-b62e-644603aafa00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:19.775367 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c282a27-e052-49a2-8c2f-f8e80d841fea None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:19.793816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 39/77] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:19.794763 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc9111-b289-439b-b62e-644603aafa00 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:19.818934 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:19.825424 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d1d62-f06e-42eb-9c1d-a300982162f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:19.826792 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d1d62-f06e-42eb-9c1d-a300982162f8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:19.827077 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d1d62-f06e-42eb-9c1d-a300982162f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:19.827373 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d1d62-f06e-42eb-9c1d-a300982162f8 None admin] PATH_INFO: `/v3/users/e13874941b204c6288bd9759b15662fb` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:19.845672 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d1d62-f06e-42eb-9c1d-a300982162f8 None admin] RBAC: Authorizing `identity:get_user(user_id=e13874941b204c6288bd9759b15662fb)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:19.847297 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d1d62-f06e-42eb-9c1d-a300982162f8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:19.848970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 39/78] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:19 2022] GET /identity/v3/users/e13874941b204c6288bd9759b15662fb => generated 275 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:19.849807 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d1d62-f06e-42eb-9c1d-a300982162f8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:19.879715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:19.926007 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc9111-b289-439b-b62e-644603aafa00 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:19.966442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d1d62-f06e-42eb-9c1d-a300982162f8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:19.967619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe2317-9e9b-40b0-9d42-5cf5cde39cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:19.968512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe2317-9e9b-40b0-9d42-5cf5cde39cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:19.968792 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe2317-9e9b-40b0-9d42-5cf5cde39cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:19.969075 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe2317-9e9b-40b0-9d42-5cf5cde39cf7 None admin] PATH_INFO: `/v3/users/e13874941b204c6288bd9759b15662fb` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:19.970397 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe2317-9e9b-40b0-9d42-5cf5cde39cf7 None admin] RBAC: Authorizing `identity:get_user(user_id=e13874941b204c6288bd9759b15662fb)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:19.971870 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe2317-9e9b-40b0-9d42-5cf5cde39cf7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:19.973603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 40/79] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:19 2022] GET /identity/v3/users/e13874941b204c6288bd9759b15662fb => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:19.974390 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe2317-9e9b-40b0-9d42-5cf5cde39cf7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:20.010931 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c915cfe-0b5f-4dc1-8385-83aa56d187e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:20.011824 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c915cfe-0b5f-4dc1-8385-83aa56d187e5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:20.012114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c915cfe-0b5f-4dc1-8385-83aa56d187e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:20.012398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c915cfe-0b5f-4dc1-8385-83aa56d187e5 None admin] PATH_INFO: `/v3/users/de745ad7f2ed48ffbb043529099e53f0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:20.013697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c915cfe-0b5f-4dc1-8385-83aa56d187e5 None admin] RBAC: Authorizing `identity:get_user(user_id=de745ad7f2ed48ffbb043529099e53f0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:20.015164 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c915cfe-0b5f-4dc1-8385-83aa56d187e5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:20.016839 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 40/80] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:19 2022] GET /identity/v3/users/de745ad7f2ed48ffbb043529099e53f0 => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:20.017637 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c915cfe-0b5f-4dc1-8385-83aa56d187e5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:20.092697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:20.109285 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe2317-9e9b-40b0-9d42-5cf5cde39cf7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:20.139154 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:20.144377 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c915cfe-0b5f-4dc1-8385-83aa56d187e5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:20.146265 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699990e-9337-4f37-bc39-10f80408aa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:20.147083 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699990e-9337-4f37-bc39-10f80408aa07 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:20.147364 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699990e-9337-4f37-bc39-10f80408aa07 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:20.147644 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699990e-9337-4f37-bc39-10f80408aa07 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:20.159593 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699990e-9337-4f37-bc39-10f80408aa07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:20.168891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a699990e-9337-4f37-bc39-10f80408aa07 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:14:20.168891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:20.168891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:20.168891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:20.168891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:20.168891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:20.168891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:20.168891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:20.168891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:20.168891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:20.168891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:20.168891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:20.168891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:20.168891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:20.168891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:20.168891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:20.170150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:20.170150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:20.170150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:20.170150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:20.170150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:20.170150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:20.170150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:20.170150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:20.170150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:20.170150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:20.170150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:20.170150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:20.170150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:20.170150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:20.170150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:20.170150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:20.171025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:20.171025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:20.171025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:20.171025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:20.171025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:20.171025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:20.171025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:20.171025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:20.171025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:20.171025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:20.171025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:14:20.171025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:14:20.171025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 41/81] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:20.171640 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699990e-9337-4f37-bc39-10f80408aa07 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:20.187391 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19614e0-4035-4660-ba01-5d79ecba91c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:20.188680 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19614e0-4035-4660-ba01-5d79ecba91c7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:20.189031 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19614e0-4035-4660-ba01-5d79ecba91c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:20.189260 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19614e0-4035-4660-ba01-5d79ecba91c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:20.190653 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19614e0-4035-4660-ba01-5d79ecba91c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:20.192201 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19614e0-4035-4660-ba01-5d79ecba91c7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:20.193981 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 41/82] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:14:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:20.194794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19614e0-4035-4660-ba01-5d79ecba91c7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:20.307055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699990e-9337-4f37-bc39-10f80408aa07 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:20.308864 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19614e0-4035-4660-ba01-5d79ecba91c7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:20.337367 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:20.343781 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d944289-d988-414c-bb11-e51cc2f3e481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:20.344463 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d944289-d988-414c-bb11-e51cc2f3e481 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:20.344749 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d944289-d988-414c-bb11-e51cc2f3e481 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:20.345028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d944289-d988-414c-bb11-e51cc2f3e481 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:20.350421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890392b3-5c74-46f9-bf26-35d963d9ce93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:20.351089 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890392b3-5c74-46f9-bf26-35d963d9ce93 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:20.351379 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890392b3-5c74-46f9-bf26-35d963d9ce93 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:20.351665 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890392b3-5c74-46f9-bf26-35d963d9ce93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:20.411608 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:20.424754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d944289-d988-414c-bb11-e51cc2f3e481 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:20.426693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d944289-d988-414c-bb11-e51cc2f3e481 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:20.428214 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890392b3-5c74-46f9-bf26-35d963d9ce93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:20.429794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890392b3-5c74-46f9-bf26-35d963d9ce93 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:20.552704 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 42/83] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:14:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:14:20.560859 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 42/84] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:14:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:14:20.560859 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890392b3-5c74-46f9-bf26-35d963d9ce93 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:20.561206 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d944289-d988-414c-bb11-e51cc2f3e481 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:20.590066 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:20.664257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:20.670919 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890392b3-5c74-46f9-bf26-35d963d9ce93 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:20.676043 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d944289-d988-414c-bb11-e51cc2f3e481 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:20.709760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75adfe14-56e4-4ed7-8401-085238df47bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:20.710617 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75adfe14-56e4-4ed7-8401-085238df47bc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:20.711030 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75adfe14-56e4-4ed7-8401-085238df47bc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:20.711892 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75adfe14-56e4-4ed7-8401-085238df47bc None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:20.712827 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75adfe14-56e4-4ed7-8401-085238df47bc None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:20.713330 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75adfe14-56e4-4ed7-8401-085238df47bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:20.714896 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75adfe14-56e4-4ed7-8401-085238df47bc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:20.716511 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda22d6-9f3a-4cef-9eac-a0cbe7483c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:20.717187 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda22d6-9f3a-4cef-9eac-a0cbe7483c40 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:20.717475 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda22d6-9f3a-4cef-9eac-a0cbe7483c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:20.717834 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda22d6-9f3a-4cef-9eac-a0cbe7483c40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:20.727985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 43/85] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:14:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:20.729102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75adfe14-56e4-4ed7-8401-085238df47bc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:20.801005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cda22d6-9f3a-4cef-9eac-a0cbe7483c40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:20.802533 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cda22d6-9f3a-4cef-9eac-a0cbe7483c40 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:20.835385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:20.850637 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75adfe14-56e4-4ed7-8401-085238df47bc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:20.888037 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2d135-9723-40b8-b8b3-71c26d0c6a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:20.888712 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2d135-9723-40b8-b8b3-71c26d0c6a83 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:20.889002 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2d135-9723-40b8-b8b3-71c26d0c6a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:20.889280 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2d135-9723-40b8-b8b3-71c26d0c6a83 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:20.890049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c2d135-9723-40b8-b8b3-71c26d0c6a83 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:20.891564 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c2d135-9723-40b8-b8b3-71c26d0c6a83 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:20.913943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:20.916235 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 44/86] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 30 18:14:20 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:14:20.917967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2d135-9723-40b8-b8b3-71c26d0c6a83 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:20.933459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 43/87] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:14:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:14:20.934352 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda22d6-9f3a-4cef-9eac-a0cbe7483c40 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:21.096727 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2d135-9723-40b8-b8b3-71c26d0c6a83 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:21.111761 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda22d6-9f3a-4cef-9eac-a0cbe7483c40 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:21.152866 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb094814-648d-4e5d-91b2-d4de1d4ac068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:21.153789 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb094814-648d-4e5d-91b2-d4de1d4ac068 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:21.154614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb094814-648d-4e5d-91b2-d4de1d4ac068 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:21.154896 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb094814-648d-4e5d-91b2-d4de1d4ac068 None admin] PATH_INFO: `/v3/projects/d5962edfa0c94d88a1056c1ed5e05058` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:21.156242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb094814-648d-4e5d-91b2-d4de1d4ac068 None admin] RBAC: Authorizing `identity:get_project(project_id=d5962edfa0c94d88a1056c1ed5e05058)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:21.157979 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb094814-648d-4e5d-91b2-d4de1d4ac068 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:21.160140 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 45/88] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:20 2022] GET /identity/v3/projects/d5962edfa0c94d88a1056c1ed5e05058 => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:21.160728 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb094814-648d-4e5d-91b2-d4de1d4ac068 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:21.173160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f9c9c-d3f6-451a-bf3d-0f2fabec7480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:21.176036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f9c9c-d3f6-451a-bf3d-0f2fabec7480 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:21.176036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f9c9c-d3f6-451a-bf3d-0f2fabec7480 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:21.176036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f9c9c-d3f6-451a-bf3d-0f2fabec7480 None admin] PATH_INFO: `/v3/projects/d5962edfa0c94d88a1056c1ed5e05058` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:21.177374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f9c9c-d3f6-451a-bf3d-0f2fabec7480 None admin] RBAC: Authorizing `identity:get_project(project_id=d5962edfa0c94d88a1056c1ed5e05058)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:21.178998 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f9c9c-d3f6-451a-bf3d-0f2fabec7480 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:21.183405 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 44/89] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:20 2022] GET /identity/v3/projects/d5962edfa0c94d88a1056c1ed5e05058 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:21.186601 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f9c9c-d3f6-451a-bf3d-0f2fabec7480 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:21.187923 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:21.307873 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:21.351346 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb094814-648d-4e5d-91b2-d4de1d4ac068 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:21.390751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f9c9c-d3f6-451a-bf3d-0f2fabec7480 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:21.407331 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5fd2d-0ffd-4be7-a3f6-be0f37e6fbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:21.407610 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5fd2d-0ffd-4be7-a3f6-be0f37e6fbc0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:21.407989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5fd2d-0ffd-4be7-a3f6-be0f37e6fbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:21.408352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5fd2d-0ffd-4be7-a3f6-be0f37e6fbc0 None admin] PATH_INFO: `/v3/users/e13874941b204c6288bd9759b15662fb` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:21.409812 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a5fd2d-0ffd-4be7-a3f6-be0f37e6fbc0 None admin] RBAC: Authorizing `identity:get_user(user_id=e13874941b204c6288bd9759b15662fb)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:21.411393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a5fd2d-0ffd-4be7-a3f6-be0f37e6fbc0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:21.413404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 46/90] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:21 2022] GET /identity/v3/users/e13874941b204c6288bd9759b15662fb => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:21.414255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5fd2d-0ffd-4be7-a3f6-be0f37e6fbc0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:21.456284 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28421ac0-68cf-4534-9b6a-658279509f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:21.457209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28421ac0-68cf-4534-9b6a-658279509f87 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:21.457497 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28421ac0-68cf-4534-9b6a-658279509f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:21.457813 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28421ac0-68cf-4534-9b6a-658279509f87 None admin] PATH_INFO: `/v3/projects/d5962edfa0c94d88a1056c1ed5e05058` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:21.459134 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28421ac0-68cf-4534-9b6a-658279509f87 None admin] RBAC: Authorizing `identity:get_project(project_id=d5962edfa0c94d88a1056c1ed5e05058)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:21.460807 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28421ac0-68cf-4534-9b6a-658279509f87 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:21.463772 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 45/91] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:21 2022] GET /identity/v3/projects/d5962edfa0c94d88a1056c1ed5e05058 => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:21.464674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28421ac0-68cf-4534-9b6a-658279509f87 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:21.532803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:21.569294 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5fd2d-0ffd-4be7-a3f6-be0f37e6fbc0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:21.611315 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a9a22-4a88-4eea-bcd9-25bbbfc4f685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:21.612406 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a9a22-4a88-4eea-bcd9-25bbbfc4f685 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:21.612909 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a9a22-4a88-4eea-bcd9-25bbbfc4f685 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:21.613406 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a9a22-4a88-4eea-bcd9-25bbbfc4f685 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:21.614494 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a9a22-4a88-4eea-bcd9-25bbbfc4f685 None admin] RBAC: Adding query filter params (role.id=3fbb71ae2de04e78b68d6d6f407ef09c, scope.project.id=d5962edfa0c94d88a1056c1ed5e05058, user.id=e13874941b204c6288bd9759b15662fb) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:21.615038 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a9a22-4a88-4eea-bcd9-25bbbfc4f685 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:21.616784 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a9a22-4a88-4eea-bcd9-25bbbfc4f685 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:21.624162 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 47/92] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:14:21 2022] GET /identity/v3/role_assignments?user.id=e13874941b204c6288bd9759b15662fb&scope.project.id=d5962edfa0c94d88a1056c1ed5e05058&role.id=3fbb71ae2de04e78b68d6d6f407ef09c => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:21.625507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a9a22-4a88-4eea-bcd9-25bbbfc4f685 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:21.630612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28421ac0-68cf-4534-9b6a-658279509f87 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:21.663564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:21.683344 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736adaf9-7d1b-4602-84ab-1a4c818faca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:21.684380 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736adaf9-7d1b-4602-84ab-1a4c818faca5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:21.684514 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736adaf9-7d1b-4602-84ab-1a4c818faca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:21.684785 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736adaf9-7d1b-4602-84ab-1a4c818faca5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:21.685673 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736adaf9-7d1b-4602-84ab-1a4c818faca5 None admin] RBAC: Adding query filter params (role.id=d5f874a3d78f4ee0ade318e1bbebf7ec, scope.project.id=d5962edfa0c94d88a1056c1ed5e05058, user.id=e13874941b204c6288bd9759b15662fb) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:21.686053 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736adaf9-7d1b-4602-84ab-1a4c818faca5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:21.687562 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736adaf9-7d1b-4602-84ab-1a4c818faca5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:21.701523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 46/93] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:14:21 2022] GET /identity/v3/role_assignments?user.id=e13874941b204c6288bd9759b15662fb&scope.project.id=d5962edfa0c94d88a1056c1ed5e05058&role.id=d5f874a3d78f4ee0ade318e1bbebf7ec => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:21.702565 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736adaf9-7d1b-4602-84ab-1a4c818faca5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:21.761407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a9a22-4a88-4eea-bcd9-25bbbfc4f685 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:21.803866 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a79c13-cf67-45ac-9899-e07bb9ac0d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:21.804912 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a79c13-cf67-45ac-9899-e07bb9ac0d6e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:21.805984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a79c13-cf67-45ac-9899-e07bb9ac0d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:21.806464 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a79c13-cf67-45ac-9899-e07bb9ac0d6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:21.836603 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:21.922936 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a79c13-cf67-45ac-9899-e07bb9ac0d6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:21.927092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a79c13-cf67-45ac-9899-e07bb9ac0d6e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:21.995717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736adaf9-7d1b-4602-84ab-1a4c818faca5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:22.066020 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef125cb-3cb4-4dbd-ba54-4f8704d58762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:22.066701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef125cb-3cb4-4dbd-ba54-4f8704d58762 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:22.066991 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef125cb-3cb4-4dbd-ba54-4f8704d58762 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:22.067272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef125cb-3cb4-4dbd-ba54-4f8704d58762 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:22.068133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef125cb-3cb4-4dbd-ba54-4f8704d58762 None admin] RBAC: Adding query filter params (role.id=0ef3687af20a443281e3cfdceca3597d, scope.project.id=d5962edfa0c94d88a1056c1ed5e05058, user.id=de745ad7f2ed48ffbb043529099e53f0) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:22.068450 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef125cb-3cb4-4dbd-ba54-4f8704d58762 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:22.073617 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef125cb-3cb4-4dbd-ba54-4f8704d58762 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:22.082413 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 48/94] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:14:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:14:22.089013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a79c13-cf67-45ac-9899-e07bb9ac0d6e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:22.089168 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 47/95] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:14:21 2022] GET /identity/v3/role_assignments?user.id=de745ad7f2ed48ffbb043529099e53f0&scope.project.id=d5962edfa0c94d88a1056c1ed5e05058&role.id=0ef3687af20a443281e3cfdceca3597d => generated 259 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:22.170085 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:22.255195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a79c13-cf67-45ac-9899-e07bb9ac0d6e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:22.294347 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5401d8-9a01-47b4-a608-26aa47cfa874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:22.295237 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5401d8-9a01-47b4-a608-26aa47cfa874 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:22.295531 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5401d8-9a01-47b4-a608-26aa47cfa874 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:22.295814 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5401d8-9a01-47b4-a608-26aa47cfa874 None admin] PATH_INFO: `/v3/projects/65dabe170c1548f3831172ac12483e04` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:22.297224 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5401d8-9a01-47b4-a608-26aa47cfa874 None admin] RBAC: Authorizing `identity:get_project(project_id=65dabe170c1548f3831172ac12483e04)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:22.298772 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5401d8-9a01-47b4-a608-26aa47cfa874 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:22.300586 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 49/96] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:22 2022] GET /identity/v3/projects/65dabe170c1548f3831172ac12483e04 => generated 312 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:22.306636 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef125cb-3cb4-4dbd-ba54-4f8704d58762 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:22.413389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:22.480054 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef125cb-3cb4-4dbd-ba54-4f8704d58762 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:22.531768 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e33f5-3c73-444b-9a40-e33e94bff095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:22.532361 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e33f5-3c73-444b-9a40-e33e94bff095 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:22.532654 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e33f5-3c73-444b-9a40-e33e94bff095 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:22.532940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e33f5-3c73-444b-9a40-e33e94bff095 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:22.533828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e33f5-3c73-444b-9a40-e33e94bff095 None admin] RBAC: Adding query filter params (role.id=3fbb71ae2de04e78b68d6d6f407ef09c, scope.project.id=65dabe170c1548f3831172ac12483e04, user.id=e13874941b204c6288bd9759b15662fb) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:22.534151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e33f5-3c73-444b-9a40-e33e94bff095 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:22.535665 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e33f5-3c73-444b-9a40-e33e94bff095 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:22.548847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 48/97] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:14:22 2022] GET /identity/v3/role_assignments?user.id=e13874941b204c6288bd9759b15662fb&scope.project.id=65dabe170c1548f3831172ac12483e04&role.id=3fbb71ae2de04e78b68d6d6f407ef09c => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:23.642323 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207810f-d1f5-4600-b97b-338f4eeff735 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:23.642832 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207810f-d1f5-4600-b97b-338f4eeff735 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:23.642906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207810f-d1f5-4600-b97b-338f4eeff735 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:23.644390 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 50/98] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:23.651639 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666c6e3-0c53-4814-a551-4f7363070480 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:23.652046 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666c6e3-0c53-4814-a551-4f7363070480 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:23.652216 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666c6e3-0c53-4814-a551-4f7363070480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:23.678592 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8666c6e3-0c53-4814-a551-4f7363070480 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:23.679804 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8666c6e3-0c53-4814-a551-4f7363070480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:23.783638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 49/99] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:23.828018 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207810f-d1f5-4600-b97b-338f4eeff735 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:23.963326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207810f-d1f5-4600-b97b-338f4eeff735 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:23.982425 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:24.006321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f8d64-e078-44d1-b8e8-b0805d73ec64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:24.007125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f8d64-e078-44d1-b8e8-b0805d73ec64 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:24.007414 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f8d64-e078-44d1-b8e8-b0805d73ec64 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:24.007692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f8d64-e078-44d1-b8e8-b0805d73ec64 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:24.018953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f8d64-e078-44d1-b8e8-b0805d73ec64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:24.030786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-063f8d64-e078-44d1-b8e8-b0805d73ec64 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 18:14:24.030786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:24.030786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:24.030786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:24.030786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:24.030786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:24.030786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:24.030786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:24.030786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:24.030786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:24.030786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:24.030786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:24.030786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:24.030786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:24.030786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:24.030786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:24.031654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:24.031654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:24.031654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:24.031654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:24.031654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:24.031654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:24.031654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:24.031654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:24.031654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:24.031654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:24.031654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:24.031654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:24.031654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:24.031654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:24.031654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:24.031654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:24.032398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:24.032398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:24.032398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:24.032398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:24.032398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:24.032398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:24.032398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:24.032398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:24.032398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:24.032398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:24.032398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 18:14:24.032398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:14:24.032398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 51/100] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:24.047411 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666c6e3-0c53-4814-a551-4f7363070480 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:24.176378 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:24.306149 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666c6e3-0c53-4814-a551-4f7363070480 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:24.374122 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e9194-83f1-41cb-b520-745ecac7049c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:24.374814 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e9194-83f1-41cb-b520-745ecac7049c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:24.375103 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e9194-83f1-41cb-b520-745ecac7049c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:24.375384 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e9194-83f1-41cb-b520-745ecac7049c None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:24.376151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e9194-83f1-41cb-b520-745ecac7049c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:24.376495 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e9194-83f1-41cb-b520-745ecac7049c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:24.377950 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e9194-83f1-41cb-b520-745ecac7049c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:24.393654 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 50/101] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:14:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:24.400202 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f8d64-e078-44d1-b8e8-b0805d73ec64 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:24.532798 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f8d64-e078-44d1-b8e8-b0805d73ec64 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:24.577283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af04b1-c0af-4adb-950e-3926812fd66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:24.578533 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af04b1-c0af-4adb-950e-3926812fd66e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:24.579032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af04b1-c0af-4adb-950e-3926812fd66e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:24.579527 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af04b1-c0af-4adb-950e-3926812fd66e None admin] PATH_INFO: `/v3/users/4ea1326eeb214c65856f79b22e345c8c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:24.595405 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af04b1-c0af-4adb-950e-3926812fd66e None admin] RBAC: Authorizing `identity:get_user(user_id=4ea1326eeb214c65856f79b22e345c8c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:24.597500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af04b1-c0af-4adb-950e-3926812fd66e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:24.600289 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 52/102] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:24 2022] GET /identity/v3/users/4ea1326eeb214c65856f79b22e345c8c => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:24.610777 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e9194-83f1-41cb-b520-745ecac7049c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:24.661605 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8b8cc-48dc-4347-9328-9f7ab0ae5a2d None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:24.662127 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8b8cc-48dc-4347-9328-9f7ab0ae5a2d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:24.662605 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8b8cc-48dc-4347-9328-9f7ab0ae5a2d None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:24.665741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 53/103] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:24.673944 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e67a0-c2f9-4c96-b8a7-436f02919073 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:24.674463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e67a0-c2f9-4c96-b8a7-436f02919073 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:24.674870 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e67a0-c2f9-4c96-b8a7-436f02919073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:24.705952 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-886e67a0-c2f9-4c96-b8a7-436f02919073 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:24.707551 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-886e67a0-c2f9-4c96-b8a7-436f02919073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:24.768056 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:24.796047 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e9194-83f1-41cb-b520-745ecac7049c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:24.823126 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:24.876520 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 54/104] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 30 18:14:24.879797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d3ff6-2fa2-44f3-8f67-77bc89027906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:24.880471 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d3ff6-2fa2-44f3-8f67-77bc89027906 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:24.880764 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d3ff6-2fa2-44f3-8f67-77bc89027906 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:24.881051 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d3ff6-2fa2-44f3-8f67-77bc89027906 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:24.922388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e67a0-c2f9-4c96-b8a7-436f02919073 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:25.020012 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d3ff6-2fa2-44f3-8f67-77bc89027906 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:25.021535 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d3ff6-2fa2-44f3-8f67-77bc89027906 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:25.177134 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e67a0-c2f9-4c96-b8a7-436f02919073 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:25.218744 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 51/105] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:14:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:14:25.220109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8510fa5-171d-4634-bace-be9da2138c05 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:25.220402 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8510fa5-171d-4634-bace-be9da2138c05 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:25.220686 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8510fa5-171d-4634-bace-be9da2138c05 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:25.222022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 52/106] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:25.229173 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147d2f9-40e2-462e-b0dc-0e7b0f03007d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:25.229465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147d2f9-40e2-462e-b0dc-0e7b0f03007d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:25.229786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147d2f9-40e2-462e-b0dc-0e7b0f03007d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:25.247966 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3eedc-8142-4835-9904-0cf970c4bec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:25.248281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1147d2f9-40e2-462e-b0dc-0e7b0f03007d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:25.248774 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3eedc-8142-4835-9904-0cf970c4bec1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:25.249061 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3eedc-8142-4835-9904-0cf970c4bec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:25.249340 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3eedc-8142-4835-9904-0cf970c4bec1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:25.250244 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1147d2f9-40e2-462e-b0dc-0e7b0f03007d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:25.253305 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3eedc-8142-4835-9904-0cf970c4bec1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:25.253695 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:25.255725 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3eedc-8142-4835-9904-0cf970c4bec1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:25.257638 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 55/107] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:14:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:25.264389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3eedc-8142-4835-9904-0cf970c4bec1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:25.305317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:25.381604 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 53/108] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:25.381604 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147d2f9-40e2-462e-b0dc-0e7b0f03007d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:25.446597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3eedc-8142-4835-9904-0cf970c4bec1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:25.500870 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1bc0c-046e-49ab-b68d-c2cbbfe77c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:25.502118 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1bc0c-046e-49ab-b68d-c2cbbfe77c02 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:25.502410 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1bc0c-046e-49ab-b68d-c2cbbfe77c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:25.502699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1bc0c-046e-49ab-b68d-c2cbbfe77c02 None admin] PATH_INFO: `/v3/projects/d5962edfa0c94d88a1056c1ed5e05058` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:25.504012 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e1bc0c-046e-49ab-b68d-c2cbbfe77c02 None admin] RBAC: Authorizing `identity:get_project(project_id=d5962edfa0c94d88a1056c1ed5e05058)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:25.505896 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e1bc0c-046e-49ab-b68d-c2cbbfe77c02 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:25.509548 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 56/109] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:25 2022] GET /identity/v3/projects/d5962edfa0c94d88a1056c1ed5e05058 => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:25.510843 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1bc0c-046e-49ab-b68d-c2cbbfe77c02 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:25.522181 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147d2f9-40e2-462e-b0dc-0e7b0f03007d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:25.562872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a47d4-aca3-4a47-bc2a-a65716ecf070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:25.563853 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a47d4-aca3-4a47-bc2a-a65716ecf070 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:25.564338 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a47d4-aca3-4a47-bc2a-a65716ecf070 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:25.564805 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a47d4-aca3-4a47-bc2a-a65716ecf070 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:25.566127 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34a47d4-aca3-4a47-bc2a-a65716ecf070 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:25.568233 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34a47d4-aca3-4a47-bc2a-a65716ecf070 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:25.580048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:25.599247 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 54/110] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 30 18:14:25 2022] POST /identity/v3/projects => generated 302 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:14:25.601897 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1cfb3d-a54c-41aa-8858-b49c319d0d4f None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:25.602530 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1cfb3d-a54c-41aa-8858-b49c319d0d4f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:25.602964 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1cfb3d-a54c-41aa-8858-b49c319d0d4f None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:25.604496 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 55/111] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:25.610674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c06665-b6ef-4d2f-acc8-37e2a7ae42ca None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:25.610674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c06665-b6ef-4d2f-acc8-37e2a7ae42ca None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:25.610674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c06665-b6ef-4d2f-acc8-37e2a7ae42ca None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:25.614098 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 56/112] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:25.615826 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c06665-b6ef-4d2f-acc8-37e2a7ae42ca None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:25.639419 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:25.647502 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1bc0c-046e-49ab-b68d-c2cbbfe77c02 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:25.699600 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57769463-4d2d-4fb2-bc46-06c8b3b9f34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:25.701245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57769463-4d2d-4fb2-bc46-06c8b3b9f34f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:25.701764 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57769463-4d2d-4fb2-bc46-06c8b3b9f34f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:25.702060 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57769463-4d2d-4fb2-bc46-06c8b3b9f34f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:25.720362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57769463-4d2d-4fb2-bc46-06c8b3b9f34f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:25.732171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-57769463-4d2d-4fb2-bc46-06c8b3b9f34f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:14:25.732171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:25.732171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:25.732171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:25.732171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:25.732171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:25.732171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:25.732171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:25.732171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:25.732171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:25.732171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:25.732171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:25.732171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:25.732171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:25.732171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:25.732171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:25.733114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:25.733114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:25.733114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:25.733114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:25.733114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:25.733114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:25.733114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:25.733114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:25.733114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:25.733114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:25.733114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:25.733114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:25.733114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:25.733114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:25.733114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:25.733114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:25.733935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:25.733935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:25.733935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:25.733935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:25.733935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:25.733935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:25.733935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:25.733935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:25.733935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:25.733935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:25.733935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:14:25.733935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:14:25.733935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 57/113] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:25.740057 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9144a0-3d4c-4f31-8171-f71342932851 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:25.740583 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9144a0-3d4c-4f31-8171-f71342932851 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:25.740989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9144a0-3d4c-4f31-8171-f71342932851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:25.759474 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ff9144a0-3d4c-4f31-8171-f71342932851 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:25.760898 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ff9144a0-3d4c-4f31-8171-f71342932851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:25.776964 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c06665-b6ef-4d2f-acc8-37e2a7ae42ca None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:25.841595 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785639d5-8040-44c8-99c2-edccbe810aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:25.842261 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785639d5-8040-44c8-99c2-edccbe810aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:25.843062 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785639d5-8040-44c8-99c2-edccbe810aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:25.843062 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785639d5-8040-44c8-99c2-edccbe810aa4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:25.843701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785639d5-8040-44c8-99c2-edccbe810aa4 None admin] RBAC: Adding query filter params (role.id=bdbac30ca21b4146bf95b3a053dff30a, scope.project.id=d5962edfa0c94d88a1056c1ed5e05058, user.id=4ea1326eeb214c65856f79b22e345c8c) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:25.844046 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785639d5-8040-44c8-99c2-edccbe810aa4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:25.847083 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785639d5-8040-44c8-99c2-edccbe810aa4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:25.859503 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 57/114] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:14:25 2022] GET /identity/v3/role_assignments?user.id=4ea1326eeb214c65856f79b22e345c8c&scope.project.id=d5962edfa0c94d88a1056c1ed5e05058&role.id=bdbac30ca21b4146bf95b3a053dff30a => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:25.859503 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f286c03-8589-4d23-99fb-b21d5d08bf59 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:25.859503 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f286c03-8589-4d23-99fb-b21d5d08bf59 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:25.859503 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f286c03-8589-4d23-99fb-b21d5d08bf59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:25.865998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 58/115] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:25.874578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9144a0-3d4c-4f31-8171-f71342932851 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:25.886329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7f286c03-8589-4d23-99fb-b21d5d08bf59 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:25.887586 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7f286c03-8589-4d23-99fb-b21d5d08bf59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:26.013710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:26.058230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 58/116] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:26.061523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f286c03-8589-4d23-99fb-b21d5d08bf59 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:26.066908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:26.111288 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9144a0-3d4c-4f31-8171-f71342932851 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:26.187330 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d63041-b348-4115-90cf-02adfca207b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:26.188032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d63041-b348-4115-90cf-02adfca207b0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:26.188321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d63041-b348-4115-90cf-02adfca207b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:26.188607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d63041-b348-4115-90cf-02adfca207b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:26.189378 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d63041-b348-4115-90cf-02adfca207b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:26.189720 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d63041-b348-4115-90cf-02adfca207b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:26.191165 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d63041-b348-4115-90cf-02adfca207b0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:26.202509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 59/117] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:14:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:26.204251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d63041-b348-4115-90cf-02adfca207b0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:26.313546 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f286c03-8589-4d23-99fb-b21d5d08bf59 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:26.360371 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8141966-a52f-40ec-9437-48340213fd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:26.362073 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8141966-a52f-40ec-9437-48340213fd97 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:26.362824 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8141966-a52f-40ec-9437-48340213fd97 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:26.363251 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8141966-a52f-40ec-9437-48340213fd97 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:26.374712 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d63041-b348-4115-90cf-02adfca207b0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:26.377158 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8141966-a52f-40ec-9437-48340213fd97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:26.386406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b8141966-a52f-40ec-9437-48340213fd97 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:14:26.386406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:26.386406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:26.386406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:26.386406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:26.386406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:26.386406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:26.386406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:26.386406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:26.386406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:26.386406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:26.386406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:26.386406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:26.386406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:26.386406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:26.386406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:26.387466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:26.387466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:26.387466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:26.387466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:26.387466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:26.387466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:26.387466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:26.387466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:26.387466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:26.387466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:26.387466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:26.387466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:26.387466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:26.387466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:26.387466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:26.387466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:26.388363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:26.388363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:26.388363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:26.388363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:26.388363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:26.388363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:26.388363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:26.388363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:26.388363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:26.388363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:26.388363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:14:26.388363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:14:26.390491 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 59/118] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 30 18:14:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:26.391465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8141966-a52f-40ec-9437-48340213fd97 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:26.418725 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bab3e-d0f2-4869-b270-ebe7b7074558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:26.419832 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bab3e-d0f2-4869-b270-ebe7b7074558 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:26.420125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bab3e-d0f2-4869-b270-ebe7b7074558 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:26.420461 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bab3e-d0f2-4869-b270-ebe7b7074558 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:26.432516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9bab3e-d0f2-4869-b270-ebe7b7074558 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:26.440970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3a9bab3e-d0f2-4869-b270-ebe7b7074558 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:14:26.440970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:26.440970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:26.440970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:26.440970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:26.440970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:26.440970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:26.440970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:26.440970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:26.440970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:26.440970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:26.440970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:26.440970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:26.440970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:26.440970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:26.440970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:26.442064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:26.442064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:26.442064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:26.442064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:26.442064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:26.442064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:26.442064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:26.442064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:26.442064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:26.442064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:26.442064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:26.442064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:26.442064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:26.442064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:26.442064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:26.442064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:26.443017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:26.443017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:26.443017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:26.443017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:26.443017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:26.443017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:26.443017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:26.443017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:26.443017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:26.443017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:26.443017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:14:26.443017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:14:26.444062 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:26.446633 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 60/119] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:14:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:26.453125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b5cde-08b8-49b6-8e60-02aed1a5b30a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:26.453750 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b5cde-08b8-49b6-8e60-02aed1a5b30a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:26.454233 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b5cde-08b8-49b6-8e60-02aed1a5b30a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:26.455808 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 61/120] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:26.456735 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b5cde-08b8-49b6-8e60-02aed1a5b30a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:26.473327 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:26.515989 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8141966-a52f-40ec-9437-48340213fd97 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:26.564687 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07cd28-c783-4ce0-a9ed-86f553536490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:26.565987 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07cd28-c783-4ce0-a9ed-86f553536490 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:26.566430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07cd28-c783-4ce0-a9ed-86f553536490 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:26.566997 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07cd28-c783-4ce0-a9ed-86f553536490 None admin] PATH_INFO: `/v3/users/e13874941b204c6288bd9759b15662fb` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:26.569065 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07cd28-c783-4ce0-a9ed-86f553536490 None admin] RBAC: Authorizing `identity:get_user(user_id=e13874941b204c6288bd9759b15662fb)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:26.572268 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07cd28-c783-4ce0-a9ed-86f553536490 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:26.575526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b5cde-08b8-49b6-8e60-02aed1a5b30a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:26.581030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 60/121] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:26 2022] GET /identity/v3/users/e13874941b204c6288bd9759b15662fb => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:26.582157 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07cd28-c783-4ce0-a9ed-86f553536490 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:26.614989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d83f5a6-e900-428c-ab44-13a249b6f961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:26.615990 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83f5a6-e900-428c-ab44-13a249b6f961 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:26.616457 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83f5a6-e900-428c-ab44-13a249b6f961 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:26.616968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83f5a6-e900-428c-ab44-13a249b6f961 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:26.618698 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d83f5a6-e900-428c-ab44-13a249b6f961 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:26.619176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d83f5a6-e900-428c-ab44-13a249b6f961 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:26.620818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d83f5a6-e900-428c-ab44-13a249b6f961 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:26.630349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 62/122] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:14:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:26.631943 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4759f-df23-40fa-96dc-8258731ab311 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:26.632435 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4759f-df23-40fa-96dc-8258731ab311 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:26.632894 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4759f-df23-40fa-96dc-8258731ab311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:26.652856 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ecb4759f-df23-40fa-96dc-8258731ab311 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:26.654470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ecb4759f-df23-40fa-96dc-8258731ab311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:26.700234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07cd28-c783-4ce0-a9ed-86f553536490 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:26.716149 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:26.726265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:26.731915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 63/123] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:26.738628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4759f-df23-40fa-96dc-8258731ab311 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:26.740145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164a16a-e3bd-468a-8538-8abdaf3f3cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:26.740840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164a16a-e3bd-468a-8538-8abdaf3f3cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:26.741139 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164a16a-e3bd-468a-8538-8abdaf3f3cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:26.741429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164a16a-e3bd-468a-8538-8abdaf3f3cb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:26.744160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164a16a-e3bd-468a-8538-8abdaf3f3cb4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:26.744160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164a16a-e3bd-468a-8538-8abdaf3f3cb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:26.745213 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164a16a-e3bd-468a-8538-8abdaf3f3cb4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:26.754284 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 61/124] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:26.755141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164a16a-e3bd-468a-8538-8abdaf3f3cb4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:26.860715 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4759f-df23-40fa-96dc-8258731ab311 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:26.876961 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164a16a-e3bd-468a-8538-8abdaf3f3cb4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:26.898295 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe9b60-7881-4100-8af2-430c27899f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:26.900166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe9b60-7881-4100-8af2-430c27899f93 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:26.900660 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe9b60-7881-4100-8af2-430c27899f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:26.901150 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe9b60-7881-4100-8af2-430c27899f93 None admin] PATH_INFO: `/v3/projects/d5962edfa0c94d88a1056c1ed5e05058` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:26.902822 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe9b60-7881-4100-8af2-430c27899f93 None admin] RBAC: Authorizing `identity:get_project(project_id=d5962edfa0c94d88a1056c1ed5e05058)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:26.904685 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe9b60-7881-4100-8af2-430c27899f93 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:26.907237 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 64/125] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:26 2022] GET /identity/v3/projects/d5962edfa0c94d88a1056c1ed5e05058 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:26.908258 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe9b60-7881-4100-8af2-430c27899f93 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:26.931351 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1e788-dad9-4040-b4d8-16f7aefdd321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:26.932623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1e788-dad9-4040-b4d8-16f7aefdd321 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:26.933124 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1e788-dad9-4040-b4d8-16f7aefdd321 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:26.933650 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1e788-dad9-4040-b4d8-16f7aefdd321 None admin] PATH_INFO: `/v3/users/e13874941b204c6288bd9759b15662fb` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:26.935137 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1e788-dad9-4040-b4d8-16f7aefdd321 None admin] RBAC: Authorizing `identity:get_user(user_id=e13874941b204c6288bd9759b15662fb)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:26.936821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1e788-dad9-4040-b4d8-16f7aefdd321 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:26.938971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 62/126] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:26 2022] GET /identity/v3/users/e13874941b204c6288bd9759b15662fb => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:26.940070 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1e788-dad9-4040-b4d8-16f7aefdd321 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:26.991329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:27.006664 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:27.058090 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe9b60-7881-4100-8af2-430c27899f93 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:27.101685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1e788-dad9-4040-b4d8-16f7aefdd321 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:27.110883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733c4bb-f04f-4237-a339-63dc72d7aa78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:27.112155 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733c4bb-f04f-4237-a339-63dc72d7aa78 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:27.112502 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733c4bb-f04f-4237-a339-63dc72d7aa78 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:27.112796 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733c4bb-f04f-4237-a339-63dc72d7aa78 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:27.126647 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1733c4bb-f04f-4237-a339-63dc72d7aa78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:27.137616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-1733c4bb-f04f-4237-a339-63dc72d7aa78 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:14:27.137616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:27.137616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:27.137616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:27.137616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:27.137616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:27.137616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:27.137616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:27.137616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:27.137616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:27.137616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:27.137616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:27.137616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:27.137616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:27.137616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:27.137616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:27.138530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:27.138530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:27.138530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:27.138530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:27.138530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:27.138530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:27.138530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:27.138530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:27.138530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:27.138530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:27.138530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:27.138530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:27.138530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:27.138530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:27.138530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:27.138530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:27.139318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:27.139318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:27.139318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:27.139318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:27.139318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:27.139318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:27.139318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:27.139318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:27.139318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:27.139318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:27.139318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:14:27.139318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:14:27.139318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 65/127] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:27.139959 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733c4bb-f04f-4237-a339-63dc72d7aa78 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:27.154029 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74f0db-26a7-4665-a64d-726523182295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:27.155780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74f0db-26a7-4665-a64d-726523182295 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:27.156286 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74f0db-26a7-4665-a64d-726523182295 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:27.156762 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74f0db-26a7-4665-a64d-726523182295 None admin] PATH_INFO: `/v3/users/de745ad7f2ed48ffbb043529099e53f0` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:27.158689 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74f0db-26a7-4665-a64d-726523182295 None admin] RBAC: Authorizing `identity:get_user(user_id=de745ad7f2ed48ffbb043529099e53f0)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:27.160979 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74f0db-26a7-4665-a64d-726523182295 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:27.163366 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 63/128] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:26 2022] GET /identity/v3/users/de745ad7f2ed48ffbb043529099e53f0 => generated 247 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:27.169119 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74f0db-26a7-4665-a64d-726523182295 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:27.282365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733c4bb-f04f-4237-a339-63dc72d7aa78 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:27.282898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74f0db-26a7-4665-a64d-726523182295 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:27.298280 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:27.318839 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:27.325548 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05078fb-2738-459e-97f5-05b1901b9ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:27.326456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05078fb-2738-459e-97f5-05b1901b9ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:27.326742 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05078fb-2738-459e-97f5-05b1901b9ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:27.327029 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05078fb-2738-459e-97f5-05b1901b9ac7 None admin] PATH_INFO: `/v3/projects/d5962edfa0c94d88a1056c1ed5e05058` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:27.328235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05078fb-2738-459e-97f5-05b1901b9ac7 None admin] RBAC: Authorizing `identity:get_project(project_id=d5962edfa0c94d88a1056c1ed5e05058)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:27.329788 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05078fb-2738-459e-97f5-05b1901b9ac7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:27.331065 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613ac03-9e9f-4f90-bc76-98cd8d2f3ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:27.331682 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 64/129] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:27 2022] GET /identity/v3/projects/d5962edfa0c94d88a1056c1ed5e05058 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:27.332301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05078fb-2738-459e-97f5-05b1901b9ac7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:27.332875 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613ac03-9e9f-4f90-bc76-98cd8d2f3ff8 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:27.333498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613ac03-9e9f-4f90-bc76-98cd8d2f3ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:27.334020 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613ac03-9e9f-4f90-bc76-98cd8d2f3ff8 None admin] PATH_INFO: `/v3/projects/d5962edfa0c94d88a1056c1ed5e05058/users/e13874941b204c6288bd9759b15662fb/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:27.338067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b613ac03-9e9f-4f90-bc76-98cd8d2f3ff8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5962edfa0c94d88a1056c1ed5e05058, user_id=e13874941b204c6288bd9759b15662fb, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:27.339835 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b613ac03-9e9f-4f90-bc76-98cd8d2f3ff8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:27.363670 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 66/130] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:14:27 2022] PUT /identity/v3/projects/d5962edfa0c94d88a1056c1ed5e05058/users/e13874941b204c6288bd9759b15662fb/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:14:27.364949 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613ac03-9e9f-4f90-bc76-98cd8d2f3ff8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:27.513617 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613ac03-9e9f-4f90-bc76-98cd8d2f3ff8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:27.569219 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000feed-6f32-4885-a3e8-8cb47d48175d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:27.571529 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000feed-6f32-4885-a3e8-8cb47d48175d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:27.572056 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000feed-6f32-4885-a3e8-8cb47d48175d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:27.572571 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000feed-6f32-4885-a3e8-8cb47d48175d None admin] PATH_INFO: `/v3/projects/d5962edfa0c94d88a1056c1ed5e05058` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:27.574143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e000feed-6f32-4885-a3e8-8cb47d48175d None admin] RBAC: Authorizing `identity:get_project(project_id=d5962edfa0c94d88a1056c1ed5e05058)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:27.576285 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e000feed-6f32-4885-a3e8-8cb47d48175d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:27.579037 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 67/131] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:27 2022] GET /identity/v3/projects/d5962edfa0c94d88a1056c1ed5e05058 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:27.580073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000feed-6f32-4885-a3e8-8cb47d48175d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:27.588782 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05078fb-2738-459e-97f5-05b1901b9ac7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:27.659422 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:27.687741 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731718b-0bf2-4f21-8b4e-d4d504ed4817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:27.688623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731718b-0bf2-4f21-8b4e-d4d504ed4817 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:27.688731 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731718b-0bf2-4f21-8b4e-d4d504ed4817 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:27.688985 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731718b-0bf2-4f21-8b4e-d4d504ed4817 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:27.689771 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731718b-0bf2-4f21-8b4e-d4d504ed4817 None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:27.690086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731718b-0bf2-4f21-8b4e-d4d504ed4817 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:27.691515 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731718b-0bf2-4f21-8b4e-d4d504ed4817 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:27.712807 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 65/132] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:14:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:27.713182 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731718b-0bf2-4f21-8b4e-d4d504ed4817 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:27.723251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000feed-6f32-4885-a3e8-8cb47d48175d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:27.770296 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:27.777556 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a2373-591c-41bd-944e-7ec8a7588945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:27.778483 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a2373-591c-41bd-944e-7ec8a7588945 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:27.778770 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a2373-591c-41bd-944e-7ec8a7588945 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:27.779052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a2373-591c-41bd-944e-7ec8a7588945 None admin] PATH_INFO: `/v3/projects/d5962edfa0c94d88a1056c1ed5e05058/users/e13874941b204c6288bd9759b15662fb/roles/d5f874a3d78f4ee0ade318e1bbebf7ec` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:27.783679 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a2373-591c-41bd-944e-7ec8a7588945 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5962edfa0c94d88a1056c1ed5e05058, user_id=e13874941b204c6288bd9759b15662fb, role_id=d5f874a3d78f4ee0ade318e1bbebf7ec)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:27.785202 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a2373-591c-41bd-944e-7ec8a7588945 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:27.809781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 68/133] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:14:27 2022] PUT /identity/v3/projects/d5962edfa0c94d88a1056c1ed5e05058/users/e13874941b204c6288bd9759b15662fb/roles/d5f874a3d78f4ee0ade318e1bbebf7ec => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:14:27.812370 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a2373-591c-41bd-944e-7ec8a7588945 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:27.923330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731718b-0bf2-4f21-8b4e-d4d504ed4817 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:28.007727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53bd496-d3a0-4b0d-a991-96ca72ab07bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:28.009163 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bd496-d3a0-4b0d-a991-96ca72ab07bb None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:28.009448 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bd496-d3a0-4b0d-a991-96ca72ab07bb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:28.009753 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bd496-d3a0-4b0d-a991-96ca72ab07bb None admin] PATH_INFO: `/v3/projects/d5962edfa0c94d88a1056c1ed5e05058/users/de745ad7f2ed48ffbb043529099e53f0/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:28.015052 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53bd496-d3a0-4b0d-a991-96ca72ab07bb None admin] RBAC: Authorizing `identity:create_grant(project_id=d5962edfa0c94d88a1056c1ed5e05058, user_id=de745ad7f2ed48ffbb043529099e53f0, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:28.016563 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53bd496-d3a0-4b0d-a991-96ca72ab07bb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:28.044270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 66/134] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:14:27 2022] PUT /identity/v3/projects/d5962edfa0c94d88a1056c1ed5e05058/users/de745ad7f2ed48ffbb043529099e53f0/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:14:28.097043 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a2373-591c-41bd-944e-7ec8a7588945 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:28.130589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:28.135547 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547b0f1-59b2-451a-b14d-747f308f25c6 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:28.135547 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547b0f1-59b2-451a-b14d-747f308f25c6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:28.135810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547b0f1-59b2-451a-b14d-747f308f25c6 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:28.139587 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 67/135] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:28.150474 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9ccb5-2317-41f2-aac1-c7468d066521 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:28.150778 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9ccb5-2317-41f2-aac1-c7468d066521 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:28.151055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9ccb5-2317-41f2-aac1-c7468d066521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:28.175251 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8aa9ccb5-2317-41f2-aac1-c7468d066521 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:28.176468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8aa9ccb5-2317-41f2-aac1-c7468d066521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:28.182406 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f844d7-0429-49e9-a520-204bd24ad72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:28.183333 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f844d7-0429-49e9-a520-204bd24ad72c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:28.183623 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f844d7-0429-49e9-a520-204bd24ad72c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:28.183909 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f844d7-0429-49e9-a520-204bd24ad72c None admin] PATH_INFO: `/v3/users/e13874941b204c6288bd9759b15662fb` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:28.185376 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f844d7-0429-49e9-a520-204bd24ad72c None admin] RBAC: Authorizing `identity:get_user(user_id=e13874941b204c6288bd9759b15662fb)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:28.186965 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f844d7-0429-49e9-a520-204bd24ad72c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:28.188787 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 69/136] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:27 2022] GET /identity/v3/users/e13874941b204c6288bd9759b15662fb => generated 275 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:28.198096 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f844d7-0429-49e9-a520-204bd24ad72c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:28.344333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 68/137] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 18:14:28.377085 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f844d7-0429-49e9-a520-204bd24ad72c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:28.417609 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a2ab4-5515-416f-90f6-d4fb51c71dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:28.418547 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a2ab4-5515-416f-90f6-d4fb51c71dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:28.418839 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a2ab4-5515-416f-90f6-d4fb51c71dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:28.419423 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a2ab4-5515-416f-90f6-d4fb51c71dc6 None admin] PATH_INFO: `/v3/projects/65dabe170c1548f3831172ac12483e04` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:28.420662 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a2ab4-5515-416f-90f6-d4fb51c71dc6 None admin] RBAC: Authorizing `identity:get_project(project_id=65dabe170c1548f3831172ac12483e04)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:28.422171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a2ab4-5515-416f-90f6-d4fb51c71dc6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:28.424331 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 70/138] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:28 2022] GET /identity/v3/projects/65dabe170c1548f3831172ac12483e04 => generated 312 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:28.434900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa9ccb5-2317-41f2-aac1-c7468d066521 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:28.435074 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a2ab4-5515-416f-90f6-d4fb51c71dc6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:28.501777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:28.504208 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:28.562035 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a2ab4-5515-416f-90f6-d4fb51c71dc6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:28.601301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa9ccb5-2317-41f2-aac1-c7468d066521 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:28.607965 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e98c6-719a-4d4e-82e7-75028dc8a022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:28.608880 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e98c6-719a-4d4e-82e7-75028dc8a022 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:28.609172 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e98c6-719a-4d4e-82e7-75028dc8a022 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:28.609456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e98c6-719a-4d4e-82e7-75028dc8a022 None admin] PATH_INFO: `/v3/projects/65dabe170c1548f3831172ac12483e04/users/e13874941b204c6288bd9759b15662fb/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:28.612427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e98c6-719a-4d4e-82e7-75028dc8a022 None admin] RBAC: Authorizing `identity:create_grant(project_id=65dabe170c1548f3831172ac12483e04, user_id=e13874941b204c6288bd9759b15662fb, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:28.614434 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e98c6-719a-4d4e-82e7-75028dc8a022 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:28.650717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713bf34-7ee9-40ec-ae36-419748fd9207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:28.651614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713bf34-7ee9-40ec-ae36-419748fd9207 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:28.651953 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713bf34-7ee9-40ec-ae36-419748fd9207 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:28.652282 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713bf34-7ee9-40ec-ae36-419748fd9207 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:28.653558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d713bf34-7ee9-40ec-ae36-419748fd9207 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:28.655054 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d713bf34-7ee9-40ec-ae36-419748fd9207 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:28.656970 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 69/139] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:14:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:28.658289 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492785c8-6e39-47a4-b688-493a70bf027d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:28.658578 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492785c8-6e39-47a4-b688-493a70bf027d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:28.658862 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492785c8-6e39-47a4-b688-493a70bf027d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:28.659529 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 71/140] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:14:28 2022] PUT /identity/v3/projects/65dabe170c1548f3831172ac12483e04/users/e13874941b204c6288bd9759b15662fb/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:14:28.660205 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 70/141] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:28.665986 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492785c8-6e39-47a4-b688-493a70bf027d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:28.666469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f35d39-a210-44cd-a4a4-2015f7ea0457 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:28.666775 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f35d39-a210-44cd-a4a4-2015f7ea0457 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:28.667067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f35d39-a210-44cd-a4a4-2015f7ea0457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:28.714776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-03f35d39-a210-44cd-a4a4-2015f7ea0457 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:28.716003 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-03f35d39-a210-44cd-a4a4-2015f7ea0457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:28.790156 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:28.797563 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492785c8-6e39-47a4-b688-493a70bf027d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:28.841350 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79c571-d815-446b-8da8-81b1b6b7ff56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:28.842149 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79c571-d815-446b-8da8-81b1b6b7ff56 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:28.842536 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79c571-d815-446b-8da8-81b1b6b7ff56 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:28.842895 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79c571-d815-446b-8da8-81b1b6b7ff56 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:28.843754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79c571-d815-446b-8da8-81b1b6b7ff56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:28.845348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79c571-d815-446b-8da8-81b1b6b7ff56 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:28.880908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 71/142] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 30 18:14:28 2022] POST /identity/v3/users => generated 283 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:14:28.923991 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 72/143] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:28.958976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79c571-d815-446b-8da8-81b1b6b7ff56 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:29.127876 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79c571-d815-446b-8da8-81b1b6b7ff56 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:29.178555 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7334ade-469c-49d7-b446-64b4d4e4b2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:29.179383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7334ade-469c-49d7-b446-64b4d4e4b2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:29.180449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7334ade-469c-49d7-b446-64b4d4e4b2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:29.180449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7334ade-469c-49d7-b446-64b4d4e4b2f2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:29.193052 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7334ade-469c-49d7-b446-64b4d4e4b2f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:29.201415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b7334ade-469c-49d7-b446-64b4d4e4b2f2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 18:14:29.201415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:29.201415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:29.201415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:29.201415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:29.201415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:29.201415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:29.201415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:29.201415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:29.201415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:29.201415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:29.201415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:29.201415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:29.201415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:29.201415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:29.201415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:29.202303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:29.202303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:29.202303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:29.202303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:29.202303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:29.202303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:29.202303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:29.202303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:29.202303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:29.202303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:29.202303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:29.202303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:29.202303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:29.202303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:29.202303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:29.202303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:29.203063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:29.203063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:29.203063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:29.203063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:29.203063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:29.203063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:29.203063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:29.203063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:29.203063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:29.203063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:29.203063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 18:14:29.203063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:14:29.203063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 72/144] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:29.208669 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f35d39-a210-44cd-a4a4-2015f7ea0457 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:29.256338 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:29.320932 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f35d39-a210-44cd-a4a4-2015f7ea0457 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:29.357472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08297e-0563-4318-9469-550c83d1f0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:29.358186 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08297e-0563-4318-9469-550c83d1f0a5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:29.358473 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08297e-0563-4318-9469-550c83d1f0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:29.358754 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08297e-0563-4318-9469-550c83d1f0a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:29.359511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08297e-0563-4318-9469-550c83d1f0a5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:29.359827 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08297e-0563-4318-9469-550c83d1f0a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:29.361218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08297e-0563-4318-9469-550c83d1f0a5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:29.370605 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 73/145] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:14:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:29.376276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7334ade-469c-49d7-b446-64b4d4e4b2f2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:29.397874 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:29.494019 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7334ade-469c-49d7-b446-64b4d4e4b2f2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:29.533168 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d44206-bb76-4989-85fa-2d102f84e454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:29.534260 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d44206-bb76-4989-85fa-2d102f84e454 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:29.534566 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d44206-bb76-4989-85fa-2d102f84e454 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:29.534848 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d44206-bb76-4989-85fa-2d102f84e454 None admin] PATH_INFO: `/v3/users/4ea1326eeb214c65856f79b22e345c8c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:29.536190 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d44206-bb76-4989-85fa-2d102f84e454 None admin] RBAC: Authorizing `identity:get_user(user_id=4ea1326eeb214c65856f79b22e345c8c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:29.537812 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d44206-bb76-4989-85fa-2d102f84e454 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:29.539789 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 73/146] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:29 2022] GET /identity/v3/users/4ea1326eeb214c65856f79b22e345c8c => generated 289 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:29.547854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08297e-0563-4318-9469-550c83d1f0a5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:29.668012 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08297e-0563-4318-9469-550c83d1f0a5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:29.670732 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:29.706556 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3109535-1dd8-4bbd-b653-6b2ff6ee0679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:29.707435 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3109535-1dd8-4bbd-b653-6b2ff6ee0679 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:29.707723 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3109535-1dd8-4bbd-b653-6b2ff6ee0679 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:29.708007 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3109535-1dd8-4bbd-b653-6b2ff6ee0679 None admin] PATH_INFO: `/v3/projects/d5962edfa0c94d88a1056c1ed5e05058` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:29.709247 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3109535-1dd8-4bbd-b653-6b2ff6ee0679 None admin] RBAC: Authorizing `identity:get_project(project_id=d5962edfa0c94d88a1056c1ed5e05058)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:29.710766 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3109535-1dd8-4bbd-b653-6b2ff6ee0679 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:29.713210 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 74/147] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:29 2022] GET /identity/v3/projects/d5962edfa0c94d88a1056c1ed5e05058 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:29.718851 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d44206-bb76-4989-85fa-2d102f84e454 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:29.826296 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:29.835459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d44206-bb76-4989-85fa-2d102f84e454 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:29.872537 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dddf79-9846-420e-8e6e-8ce457a58d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:29.873437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dddf79-9846-420e-8e6e-8ce457a58d18 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:29.873760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dddf79-9846-420e-8e6e-8ce457a58d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:29.874040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dddf79-9846-420e-8e6e-8ce457a58d18 None admin] PATH_INFO: `/v3/projects/d5962edfa0c94d88a1056c1ed5e05058/users/4ea1326eeb214c65856f79b22e345c8c/roles/bdbac30ca21b4146bf95b3a053dff30a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:29.876067 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dddf79-9846-420e-8e6e-8ce457a58d18 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5962edfa0c94d88a1056c1ed5e05058, user_id=4ea1326eeb214c65856f79b22e345c8c, role_id=bdbac30ca21b4146bf95b3a053dff30a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:29.877542 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dddf79-9846-420e-8e6e-8ce457a58d18 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:29.894864 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 74/148] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:14:29 2022] PUT /identity/v3/projects/d5962edfa0c94d88a1056c1ed5e05058/users/4ea1326eeb214c65856f79b22e345c8c/roles/bdbac30ca21b4146bf95b3a053dff30a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:14:30.636773 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7cdfc-31cc-447f-8a37-2e3acf906d25 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:30.637850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7cdfc-31cc-447f-8a37-2e3acf906d25 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:30.638358 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7cdfc-31cc-447f-8a37-2e3acf906d25 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:30.639984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 75/149] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:30.647677 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b73da8-cba7-4f16-b338-3e70d848bf84 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:30.649742 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b73da8-cba7-4f16-b338-3e70d848bf84 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:30.650552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b73da8-cba7-4f16-b338-3e70d848bf84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:30.679001 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-83b73da8-cba7-4f16-b338-3e70d848bf84 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:30.680779 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-83b73da8-cba7-4f16-b338-3e70d848bf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:30.777110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 75/150] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 18:14:30.826267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7cdfc-31cc-447f-8a37-2e3acf906d25 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:30.968656 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7cdfc-31cc-447f-8a37-2e3acf906d25 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:31.006903 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76178675-acc5-47c8-a4e4-a5ef7e44eaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:31.008258 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76178675-acc5-47c8-a4e4-a5ef7e44eaae None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:31.008639 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76178675-acc5-47c8-a4e4-a5ef7e44eaae None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:31.009130 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76178675-acc5-47c8-a4e4-a5ef7e44eaae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:31.020626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76178675-acc5-47c8-a4e4-a5ef7e44eaae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:31.029052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-76178675-acc5-47c8-a4e4-a5ef7e44eaae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:14:31.029052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:31.029052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:31.029052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:31.029052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:31.029052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:31.029052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:31.029052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:31.029052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:31.029052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:31.029052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:31.029052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:31.029052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:31.029052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:31.029052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:31.029052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:31.030021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:31.030021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:31.030021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:31.030021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:31.030021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:31.030021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:31.030021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:31.030021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:31.030021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:31.030021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:31.030021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:31.030021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:31.030021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:31.030021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:31.030021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:31.030021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:31.030803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:31.030803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:31.030803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:31.030803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:31.030803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:31.030803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:31.030803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:31.030803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:31.030803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:31.030803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:31.030803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:14:31.030803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:14:31.030803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 76/151] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:31.036352 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b73da8-cba7-4f16-b338-3e70d848bf84 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:31.120170 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:31.146355 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b73da8-cba7-4f16-b338-3e70d848bf84 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:31.184382 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45295c-1ff3-4d51-8304-575c2103fb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:31.185063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45295c-1ff3-4d51-8304-575c2103fb01 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:31.185346 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45295c-1ff3-4d51-8304-575c2103fb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:31.185644 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45295c-1ff3-4d51-8304-575c2103fb01 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:31.186402 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45295c-1ff3-4d51-8304-575c2103fb01 None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:31.186712 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45295c-1ff3-4d51-8304-575c2103fb01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:31.188099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45295c-1ff3-4d51-8304-575c2103fb01 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:31.197834 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 76/152] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:14:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:31.203343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76178675-acc5-47c8-a4e4-a5ef7e44eaae None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:31.237495 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:31.325314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76178675-acc5-47c8-a4e4-a5ef7e44eaae None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:31.372004 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceabb10-6f90-4e7b-9037-f64ea8c1cf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:31.372896 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceabb10-6f90-4e7b-9037-f64ea8c1cf6b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:31.373182 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceabb10-6f90-4e7b-9037-f64ea8c1cf6b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:31.373463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceabb10-6f90-4e7b-9037-f64ea8c1cf6b None admin] PATH_INFO: `/v3/users/e13874941b204c6288bd9759b15662fb` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:31.374706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceabb10-6f90-4e7b-9037-f64ea8c1cf6b None admin] RBAC: Authorizing `identity:get_user(user_id=e13874941b204c6288bd9759b15662fb)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:31.376173 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceabb10-6f90-4e7b-9037-f64ea8c1cf6b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:31.377852 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 77/153] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:31 2022] GET /identity/v3/users/e13874941b204c6288bd9759b15662fb => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:31.383919 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45295c-1ff3-4d51-8304-575c2103fb01 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:31.493820 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a9cb8-e489-44c9-a15b-b4c26eeb966b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:31.494145 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a9cb8-e489-44c9-a15b-b4c26eeb966b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:31.494310 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a9cb8-e489-44c9-a15b-b4c26eeb966b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:31.495846 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 78/154] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:14:31.497314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d7c18-08af-4420-91bc-3d48cc473555 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:31.497619 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d7c18-08af-4420-91bc-3d48cc473555 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:31.497908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d7c18-08af-4420-91bc-3d48cc473555 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:31.499136 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 79/155] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:14:31.499460 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45295c-1ff3-4d51-8304-575c2103fb01 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:31.504459 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd5800-4776-4f20-9142-0bbc5b9f55a2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:31.504747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd5800-4776-4f20-9142-0bbc5b9f55a2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:31.505027 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd5800-4776-4f20-9142-0bbc5b9f55a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:31.526523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-60cd5800-4776-4f20-9142-0bbc5b9f55a2 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:31.527901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-60cd5800-4776-4f20-9142-0bbc5b9f55a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:31.565615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edbf52b-e17d-43df-a4e1-d8d9ef207afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:31.566290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edbf52b-e17d-43df-a4e1-d8d9ef207afb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:31.566579 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edbf52b-e17d-43df-a4e1-d8d9ef207afb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:31.566860 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edbf52b-e17d-43df-a4e1-d8d9ef207afb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:31.583845 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:31.668285 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 80/156] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 18:14:31.669908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5356761-4662-46ae-866b-f57fc05a558e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:31.670202 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5356761-4662-46ae-866b-f57fc05a558e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:31.670490 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5356761-4662-46ae-866b-f57fc05a558e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:31.701606 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e5356761-4662-46ae-866b-f57fc05a558e None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:31.702825 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e5356761-4662-46ae-866b-f57fc05a558e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:31.736835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edbf52b-e17d-43df-a4e1-d8d9ef207afb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:31.738373 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edbf52b-e17d-43df-a4e1-d8d9ef207afb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:31.755145 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:31.826023 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 81/157] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:31.827189 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5356761-4662-46ae-866b-f57fc05a558e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:31.944519 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 77/158] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:14:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:14:31.949877 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edbf52b-e17d-43df-a4e1-d8d9ef207afb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:32.083321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:32.114130 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5356761-4662-46ae-866b-f57fc05a558e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:32.204718 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d50a8-0d43-4327-8639-1d760a9e1259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:32.205910 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d50a8-0d43-4327-8639-1d760a9e1259 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:32.206432 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d50a8-0d43-4327-8639-1d760a9e1259 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:32.206994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d50a8-0d43-4327-8639-1d760a9e1259 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:32.236028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d50a8-0d43-4327-8639-1d760a9e1259 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:32.247642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-941d50a8-0d43-4327-8639-1d760a9e1259 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:14:32.247642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:32.247642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:32.247642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:32.247642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:32.247642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:32.247642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:32.247642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:32.247642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:32.247642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:32.247642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:32.247642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:32.247642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:32.247642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:32.247642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:32.247642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:32.248678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:32.248678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:32.248678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:32.248678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:32.248678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:32.248678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:32.248678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:32.248678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:32.248678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:32.248678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:32.248678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:32.248678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:32.248678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:32.248678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:32.248678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:32.248678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:32.249485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:32.249485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:32.249485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:32.249485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:32.249485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:32.249485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:32.249485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:32.249485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:32.249485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:32.249485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:32.249485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:14:32.249485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:14:32.249485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 82/159] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:14:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:32.254056 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d50a8-0d43-4327-8639-1d760a9e1259 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:32.260279 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edbf52b-e17d-43df-a4e1-d8d9ef207afb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:32.306133 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:32.333609 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff2adf-8008-4903-819b-238e863badc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:32.334432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff2adf-8008-4903-819b-238e863badc3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:32.334714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff2adf-8008-4903-819b-238e863badc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:32.334992 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff2adf-8008-4903-819b-238e863badc3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:32.360428 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff2adf-8008-4903-819b-238e863badc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:32.373413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-fdff2adf-8008-4903-819b-238e863badc3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:14:32.373413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:32.373413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:32.373413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:32.373413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:32.373413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:32.373413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:32.373413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:32.373413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:32.373413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:32.373413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:32.373413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:32.373413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:32.373413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:32.373413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:32.373413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:32.375294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:32.375294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:32.375294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:32.375294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:32.375294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:32.375294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:32.375294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:32.375294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:32.375294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:32.375294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:32.375294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:32.375294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:32.375294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:32.375294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:32.375294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:32.375294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:32.376847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:32.376847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:32.376847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:32.376847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:32.376847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:32.376847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:32.376847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:32.376847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:32.376847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:32.376847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:32.376847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:14:32.376847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:14:32.379280 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 78/160] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 30 18:14:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:32.381825 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff2adf-8008-4903-819b-238e863badc3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:32.507601 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d50a8-0d43-4327-8639-1d760a9e1259 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:32.585453 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f287d6-b0cd-426b-a8d9-5dece964280c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:32.586901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f287d6-b0cd-426b-a8d9-5dece964280c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:32.587189 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f287d6-b0cd-426b-a8d9-5dece964280c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:32.587478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f287d6-b0cd-426b-a8d9-5dece964280c None admin] PATH_INFO: `/v3/projects/d5962edfa0c94d88a1056c1ed5e05058` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:32.590523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f287d6-b0cd-426b-a8d9-5dece964280c None admin] RBAC: Authorizing `identity:get_project(project_id=d5962edfa0c94d88a1056c1ed5e05058)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:32.592157 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f287d6-b0cd-426b-a8d9-5dece964280c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:32.593973 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 83/161] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:32 2022] GET /identity/v3/projects/d5962edfa0c94d88a1056c1ed5e05058 => generated 298 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:32.598465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f287d6-b0cd-426b-a8d9-5dece964280c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:32.664184 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:32.683481 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff2adf-8008-4903-819b-238e863badc3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:32.797184 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80592092-98c9-448c-9d08-7c436b8b1962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:32.799815 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80592092-98c9-448c-9d08-7c436b8b1962 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:32.806341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80592092-98c9-448c-9d08-7c436b8b1962 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:32.807415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80592092-98c9-448c-9d08-7c436b8b1962 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:32.809123 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80592092-98c9-448c-9d08-7c436b8b1962 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:32.809980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80592092-98c9-448c-9d08-7c436b8b1962 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:32.813018 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80592092-98c9-448c-9d08-7c436b8b1962 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:32.814041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:32.845974 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 79/162] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 466 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:32.848060 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f2391-6476-45e8-ac9d-6eef129a94bc None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:32.848982 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f2391-6476-45e8-ac9d-6eef129a94bc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:32.849470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f2391-6476-45e8-ac9d-6eef129a94bc None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:32.851040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 80/163] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:32.852265 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f2391-6476-45e8-ac9d-6eef129a94bc None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:32.855648 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f287d6-b0cd-426b-a8d9-5dece964280c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:32.939241 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05430b38-add0-4891-b9eb-3b9f2eb3a047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:32.940367 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05430b38-add0-4891-b9eb-3b9f2eb3a047 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:32.940860 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05430b38-add0-4891-b9eb-3b9f2eb3a047 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:32.941367 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05430b38-add0-4891-b9eb-3b9f2eb3a047 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:32.943676 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05430b38-add0-4891-b9eb-3b9f2eb3a047 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:32.944341 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05430b38-add0-4891-b9eb-3b9f2eb3a047 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:32.948564 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05430b38-add0-4891-b9eb-3b9f2eb3a047 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:32.959079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 84/164] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:14:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:32.960917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c135139-b7b9-48be-a1db-729086a1dc90 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:32.961430 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c135139-b7b9-48be-a1db-729086a1dc90 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:32.961953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c135139-b7b9-48be-a1db-729086a1dc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:33.001912 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8c135139-b7b9-48be-a1db-729086a1dc90 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:33.003494 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8c135139-b7b9-48be-a1db-729086a1dc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:33.030218 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f2391-6476-45e8-ac9d-6eef129a94bc None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:33.068103 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c8abf-8024-4657-a9d0-d770303888f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:33.072824 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c8abf-8024-4657-a9d0-d770303888f2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:33.072824 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c8abf-8024-4657-a9d0-d770303888f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:33.072824 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c8abf-8024-4657-a9d0-d770303888f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:33.072824 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c8abf-8024-4657-a9d0-d770303888f2 None admin] RBAC: Adding query filter params (role.id=3fbb71ae2de04e78b68d6d6f407ef09c, scope.project.id=d5962edfa0c94d88a1056c1ed5e05058, user.id=e13874941b204c6288bd9759b15662fb) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:33.072824 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c8abf-8024-4657-a9d0-d770303888f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:33.072824 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c8abf-8024-4657-a9d0-d770303888f2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:33.079129 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 81/165] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:14:32 2022] GET /identity/v3/role_assignments?user.id=e13874941b204c6288bd9759b15662fb&scope.project.id=d5962edfa0c94d88a1056c1ed5e05058&role.id=3fbb71ae2de04e78b68d6d6f407ef09c => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:33.080880 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c8abf-8024-4657-a9d0-d770303888f2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:33.158064 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:33.172598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 85/166] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:33.174005 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c135139-b7b9-48be-a1db-729086a1dc90 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:33.327590 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:33.376658 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c135139-b7b9-48be-a1db-729086a1dc90 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:33.400211 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c8abf-8024-4657-a9d0-d770303888f2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:33.456788 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31eb5c-a63b-49ec-9853-4f55a44839ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:33.457842 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31eb5c-a63b-49ec-9853-4f55a44839ef None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:33.458794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31eb5c-a63b-49ec-9853-4f55a44839ef None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:33.459093 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31eb5c-a63b-49ec-9853-4f55a44839ef None admin] PATH_INFO: `/v3/users/e13874941b204c6288bd9759b15662fb` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:33.462412 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31eb5c-a63b-49ec-9853-4f55a44839ef None admin] RBAC: Authorizing `identity:get_user(user_id=e13874941b204c6288bd9759b15662fb)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:33.463900 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31eb5c-a63b-49ec-9853-4f55a44839ef None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:33.465844 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 86/167] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:33 2022] GET /identity/v3/users/e13874941b204c6288bd9759b15662fb => generated 275 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:33.466454 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31eb5c-a63b-49ec-9853-4f55a44839ef None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:33.507414 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150b155-cfb2-4953-abd1-2baed7d7d24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:33.508884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150b155-cfb2-4953-abd1-2baed7d7d24a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:33.509370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150b155-cfb2-4953-abd1-2baed7d7d24a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:33.509694 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150b155-cfb2-4953-abd1-2baed7d7d24a None admin] PATH_INFO: `/v3/users/de745ad7f2ed48ffbb043529099e53f0` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:33.514394 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1150b155-cfb2-4953-abd1-2baed7d7d24a None admin] RBAC: Authorizing `identity:get_user(user_id=de745ad7f2ed48ffbb043529099e53f0)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:33.515917 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1150b155-cfb2-4953-abd1-2baed7d7d24a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:33.517674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 82/168] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:33 2022] GET /identity/v3/users/de745ad7f2ed48ffbb043529099e53f0 => generated 247 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:33.518474 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150b155-cfb2-4953-abd1-2baed7d7d24a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:33.667386 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31eb5c-a63b-49ec-9853-4f55a44839ef None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:33.708998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405ce3d-f942-4b97-be32-d861c09b934f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:33.709765 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405ce3d-f942-4b97-be32-d861c09b934f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:33.710204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405ce3d-f942-4b97-be32-d861c09b934f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:33.710485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405ce3d-f942-4b97-be32-d861c09b934f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:33.725102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d405ce3d-f942-4b97-be32-d861c09b934f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:33.737597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d405ce3d-f942-4b97-be32-d861c09b934f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:14:33.737597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:33.737597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:33.737597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:33.737597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:33.737597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:33.737597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:33.737597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:33.737597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:33.737597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:33.737597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:33.737597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:33.737597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:33.737597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:33.737597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:33.737597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:33.738521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:33.738521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:33.738521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:33.738521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:33.738521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:33.738521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:33.738521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:33.738521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:33.738521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:33.738521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:33.738521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:33.738521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:33.738521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:33.738521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:33.738521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:33.738521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:33.739313 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:33.739313 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:33.739313 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:33.739313 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:33.739313 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:33.739313 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:33.739313 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:33.739313 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:33.739313 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:33.739313 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:33.739313 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:14:33.739313 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:14:33.739313 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 87/169] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:33.741197 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405ce3d-f942-4b97-be32-d861c09b934f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:33.762597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:33.780344 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:33.800549 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150b155-cfb2-4953-abd1-2baed7d7d24a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:33.876594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e601e0b-2717-4a46-b7b5-2ab65cd5f9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:33.877264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e601e0b-2717-4a46-b7b5-2ab65cd5f9bd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:33.877533 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e601e0b-2717-4a46-b7b5-2ab65cd5f9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:33.877845 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e601e0b-2717-4a46-b7b5-2ab65cd5f9bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:33.945791 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405ce3d-f942-4b97-be32-d861c09b934f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:34.020741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44233d-7eb2-4106-9e21-0d5d3783e0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:34.024270 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44233d-7eb2-4106-9e21-0d5d3783e0aa None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:34.024270 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44233d-7eb2-4106-9e21-0d5d3783e0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:34.024270 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44233d-7eb2-4106-9e21-0d5d3783e0aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:34.033604 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e601e0b-2717-4a46-b7b5-2ab65cd5f9bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:34.033604 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e601e0b-2717-4a46-b7b5-2ab65cd5f9bd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:34.162911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd44233d-7eb2-4106-9e21-0d5d3783e0aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:34.165737 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd44233d-7eb2-4106-9e21-0d5d3783e0aa None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:34.178610 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:34.220044 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:34.235850 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 83/170] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:14:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 717 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:14:34.236903 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e601e0b-2717-4a46-b7b5-2ab65cd5f9bd None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:34.302606 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 88/171] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:14:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:14:34.306659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a73d2-ac92-4f00-806a-050e1ad66d71 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:34.307280 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a73d2-ac92-4f00-806a-050e1ad66d71 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:34.307879 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a73d2-ac92-4f00-806a-050e1ad66d71 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:34.310641 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 89/172] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:34.313117 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332db25-dfba-4516-98d6-9e1a98e76ae4 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:34.313795 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332db25-dfba-4516-98d6-9e1a98e76ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:34.314401 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332db25-dfba-4516-98d6-9e1a98e76ae4 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:34.317036 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 90/173] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:34.318526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332db25-dfba-4516-98d6-9e1a98e76ae4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:34.391130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e601e0b-2717-4a46-b7b5-2ab65cd5f9bd None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:34.431783 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059e1f7-5e99-467e-9d0d-753298b0ff0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:34.432467 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059e1f7-5e99-467e-9d0d-753298b0ff0c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:34.432748 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059e1f7-5e99-467e-9d0d-753298b0ff0c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:34.433026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059e1f7-5e99-467e-9d0d-753298b0ff0c None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:34.433821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059e1f7-5e99-467e-9d0d-753298b0ff0c None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:34.434137 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059e1f7-5e99-467e-9d0d-753298b0ff0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:34.435550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059e1f7-5e99-467e-9d0d-753298b0ff0c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:34.443946 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332db25-dfba-4516-98d6-9e1a98e76ae4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:34.444803 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 84/174] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:14:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:34.446890 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344a363-2975-45ae-947f-1733eae101d9 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:34.447189 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344a363-2975-45ae-947f-1733eae101d9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:34.447474 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344a363-2975-45ae-947f-1733eae101d9 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:34.449036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 85/175] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:34.453742 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344a363-2975-45ae-947f-1733eae101d9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:34.458168 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:34.492764 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8ecb7-8c89-4413-9114-b8bde2cb1ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:34.493884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8ecb7-8c89-4413-9114-b8bde2cb1ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:34.494223 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8ecb7-8c89-4413-9114-b8bde2cb1ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:34.494692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8ecb7-8c89-4413-9114-b8bde2cb1ac6 None admin] PATH_INFO: `/v3/projects/d5962edfa0c94d88a1056c1ed5e05058` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:34.496374 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b8ecb7-8c89-4413-9114-b8bde2cb1ac6 None admin] RBAC: Authorizing `identity:get_project(project_id=d5962edfa0c94d88a1056c1ed5e05058)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:34.498444 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b8ecb7-8c89-4413-9114-b8bde2cb1ac6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:34.500410 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 91/176] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:34 2022] GET /identity/v3/projects/d5962edfa0c94d88a1056c1ed5e05058 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:34.505375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25952a-9de5-4636-8648-d0723c2a6541 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:34.505375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25952a-9de5-4636-8648-d0723c2a6541 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:34.505375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25952a-9de5-4636-8648-d0723c2a6541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:34.516222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:34.530558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ac25952a-9de5-4636-8648-d0723c2a6541 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:34.532714 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ac25952a-9de5-4636-8648-d0723c2a6541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:34.576159 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344a363-2975-45ae-947f-1733eae101d9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:34.619320 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 92/177] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:34.620656 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980104d-25fd-4606-891b-318387cae60f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:34.620949 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980104d-25fd-4606-891b-318387cae60f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:34.621311 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980104d-25fd-4606-891b-318387cae60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:34.629570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e09d2-1b6f-4cf0-981b-70054e4dc3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:34.630598 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e09d2-1b6f-4cf0-981b-70054e4dc3db None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:34.630895 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e09d2-1b6f-4cf0-981b-70054e4dc3db None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:34.631172 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e09d2-1b6f-4cf0-981b-70054e4dc3db None admin] PATH_INFO: `/v3/projects/d5962edfa0c94d88a1056c1ed5e05058` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:34.633470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e09d2-1b6f-4cf0-981b-70054e4dc3db None admin] RBAC: Authorizing `identity:get_project(project_id=d5962edfa0c94d88a1056c1ed5e05058)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:34.635820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e09d2-1b6f-4cf0-981b-70054e4dc3db None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:34.638725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 86/178] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:34 2022] GET /identity/v3/projects/d5962edfa0c94d88a1056c1ed5e05058 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:34.638858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f980104d-25fd-4606-891b-318387cae60f None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:34.640067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f980104d-25fd-4606-891b-318387cae60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:34.640642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fd46b-cbe7-46dc-ab63-7d8ed8d76bd9 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:34.641064 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fd46b-cbe7-46dc-ab63-7d8ed8d76bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:34.641472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fd46b-cbe7-46dc-ab63-7d8ed8d76bd9 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:34.644215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 87/179] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:34.644918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09fd46b-cbe7-46dc-ab63-7d8ed8d76bd9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:34.722182 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 93/180] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:34.723638 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873121e-8189-4207-88b2-ff9f96ae4663 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:34.723881 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873121e-8189-4207-88b2-ff9f96ae4663 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:34.724165 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873121e-8189-4207-88b2-ff9f96ae4663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:34.751389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f873121e-8189-4207-88b2-ff9f96ae4663 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:34.752635 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f873121e-8189-4207-88b2-ff9f96ae4663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:34.768322 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09fd46b-cbe7-46dc-ab63-7d8ed8d76bd9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:34.784228 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:34.789750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:34.805339 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6aa4dd-c1e1-412e-a37e-4db2d9717765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:34.806241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6aa4dd-c1e1-412e-a37e-4db2d9717765 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:34.806520 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6aa4dd-c1e1-412e-a37e-4db2d9717765 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:34.806792 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6aa4dd-c1e1-412e-a37e-4db2d9717765 None admin] PATH_INFO: `/v3/users/e13874941b204c6288bd9759b15662fb` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:34.808005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6aa4dd-c1e1-412e-a37e-4db2d9717765 None admin] RBAC: Authorizing `identity:get_user(user_id=e13874941b204c6288bd9759b15662fb)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:34.809483 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6aa4dd-c1e1-412e-a37e-4db2d9717765 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:34.811180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 88/181] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:34 2022] GET /identity/v3/users/e13874941b204c6288bd9759b15662fb => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:34.811956 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6aa4dd-c1e1-412e-a37e-4db2d9717765 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:34.839296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 94/182] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:34.841102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873121e-8189-4207-88b2-ff9f96ae4663 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:34.942179 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6aa4dd-c1e1-412e-a37e-4db2d9717765 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:34.986065 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18891ec5-639e-4338-a967-e36e961c2740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:34.986725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18891ec5-639e-4338-a967-e36e961c2740 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:34.987008 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18891ec5-639e-4338-a967-e36e961c2740 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:34.987286 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18891ec5-639e-4338-a967-e36e961c2740 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:34.988130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18891ec5-639e-4338-a967-e36e961c2740 None admin] RBAC: Adding query filter params (role.id=d5f874a3d78f4ee0ade318e1bbebf7ec, scope.project.id=d5962edfa0c94d88a1056c1ed5e05058, user.id=e13874941b204c6288bd9759b15662fb) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:34.988445 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18891ec5-639e-4338-a967-e36e961c2740 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:34.989310 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873121e-8189-4207-88b2-ff9f96ae4663 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:34.989942 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18891ec5-639e-4338-a967-e36e961c2740 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:34.996745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 89/183] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:14:34 2022] GET /identity/v3/role_assignments?user.id=e13874941b204c6288bd9759b15662fb&scope.project.id=d5962edfa0c94d88a1056c1ed5e05058&role.id=d5f874a3d78f4ee0ade318e1bbebf7ec => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:34.998239 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bf94b-1b6a-40da-83b2-b5fbf470a7d1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:34.998535 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bf94b-1b6a-40da-83b2-b5fbf470a7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:34.998826 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bf94b-1b6a-40da-83b2-b5fbf470a7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:35.019868 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-439bf94b-1b6a-40da-83b2-b5fbf470a7d1 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:35.021719 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-439bf94b-1b6a-40da-83b2-b5fbf470a7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:35.048387 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad0982-b63a-44a7-b8b9-ada3ea423638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:35.049069 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad0982-b63a-44a7-b8b9-ada3ea423638 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:35.049354 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad0982-b63a-44a7-b8b9-ada3ea423638 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:35.049675 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad0982-b63a-44a7-b8b9-ada3ea423638 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:35.050544 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad0982-b63a-44a7-b8b9-ada3ea423638 None admin] RBAC: Adding query filter params (role.id=0ef3687af20a443281e3cfdceca3597d, scope.project.id=d5962edfa0c94d88a1056c1ed5e05058, user.id=de745ad7f2ed48ffbb043529099e53f0) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:35.050883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad0982-b63a-44a7-b8b9-ada3ea423638 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:35.052415 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad0982-b63a-44a7-b8b9-ada3ea423638 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:35.060926 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 95/184] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:14:34 2022] GET /identity/v3/role_assignments?user.id=de745ad7f2ed48ffbb043529099e53f0&scope.project.id=d5962edfa0c94d88a1056c1ed5e05058&role.id=0ef3687af20a443281e3cfdceca3597d => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:35.061880 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad0982-b63a-44a7-b8b9-ada3ea423638 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:35.109895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:35.117835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:35.169775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 90/185] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:35.169775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439bf94b-1b6a-40da-83b2-b5fbf470a7d1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:35.246781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad0982-b63a-44a7-b8b9-ada3ea423638 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:35.296331 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a4859-1845-44fc-9582-7ad7787318bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:35.297550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a4859-1845-44fc-9582-7ad7787318bd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:35.298066 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a4859-1845-44fc-9582-7ad7787318bd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:35.298347 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a4859-1845-44fc-9582-7ad7787318bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:35.314024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a4859-1845-44fc-9582-7ad7787318bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:35.320621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439bf94b-1b6a-40da-83b2-b5fbf470a7d1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:35.322215 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4f6a4859-1845-44fc-9582-7ad7787318bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:14:35.322215 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:35.322215 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:35.322215 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:35.322215 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:35.322215 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:35.322215 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:35.322215 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:35.322215 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:35.322215 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:35.322215 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:35.322215 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:35.322215 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:35.322215 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:35.322215 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:35.322215 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:35.323094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:35.323094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:35.323094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:35.323094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:35.323094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:35.323094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:35.323094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:35.323094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:35.323094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:35.323094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:35.323094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:35.323094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:35.323094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:35.323094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:35.323094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:35.323094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:35.323878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:35.323878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:35.323878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:35.323878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:35.323878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:35.323878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:35.323878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:35.323878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:35.323878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:35.323878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:35.323878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:14:35.323878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:14:35.323878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 96/186] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:14:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:35.324527 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a4859-1845-44fc-9582-7ad7787318bd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:35.375615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bb7bd-f9ed-4576-bc96-0a43e667ee83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:35.376340 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bb7bd-f9ed-4576-bc96-0a43e667ee83 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:35.376625 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bb7bd-f9ed-4576-bc96-0a43e667ee83 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:35.376904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bb7bd-f9ed-4576-bc96-0a43e667ee83 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:35.389287 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665bb7bd-f9ed-4576-bc96-0a43e667ee83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:35.397031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-665bb7bd-f9ed-4576-bc96-0a43e667ee83 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 18:14:35.397031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:35.397031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:35.397031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:35.397031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:35.397031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:35.397031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:35.397031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:35.397031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:35.397031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:35.397031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:35.397031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:35.397031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:35.397031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:35.397031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:35.397031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:35.399200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:35.399200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:35.399200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:35.399200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:35.399200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:35.399200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:35.399200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:35.399200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:35.399200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:35.399200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:35.399200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:35.399200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:35.399200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:35.399200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:35.399200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:35.399200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:35.400451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:35.400451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:35.400451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:35.400451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:35.400451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:35.400451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:35.400451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:35.400451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:35.400451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:35.400451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:35.400451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 18:14:35.400451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:14:35.400451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 91/187] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:35.400451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bb7bd-f9ed-4576-bc96-0a43e667ee83 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:35.438067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:35.450052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a4859-1845-44fc-9582-7ad7787318bd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:35.468922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:35.489265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c934cf4-4b38-402c-b71b-11d1db3fb6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:35.489945 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c934cf4-4b38-402c-b71b-11d1db3fb6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:35.490227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c934cf4-4b38-402c-b71b-11d1db3fb6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:35.490501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c934cf4-4b38-402c-b71b-11d1db3fb6c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:35.533688 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bb7bd-f9ed-4576-bc96-0a43e667ee83 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:35.571822 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c934cf4-4b38-402c-b71b-11d1db3fb6c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:35.572555 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ffa9f-0a3c-4f79-bf60-7667c8219307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:35.573335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c934cf4-4b38-402c-b71b-11d1db3fb6c8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:35.573523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ffa9f-0a3c-4f79-bf60-7667c8219307 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:35.573931 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ffa9f-0a3c-4f79-bf60-7667c8219307 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:35.574357 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ffa9f-0a3c-4f79-bf60-7667c8219307 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:35.586451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ffa9f-0a3c-4f79-bf60-7667c8219307 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:35.594615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-006ffa9f-0a3c-4f79-bf60-7667c8219307 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:14:35.594615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:35.594615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:35.594615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:35.594615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:35.594615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:35.594615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:35.594615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:35.594615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:35.594615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:35.594615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:35.594615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:35.594615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:35.594615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:35.594615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:35.594615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:35.595729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:35.595729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:35.595729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:35.595729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:35.595729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:35.595729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:35.595729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:35.595729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:35.595729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:35.595729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:35.595729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:35.595729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:35.595729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:35.595729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:35.595729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:35.595729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:35.596615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:35.596615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:35.596615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:35.596615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:35.596615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:35.596615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:35.596615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:35.596615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:35.596615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:35.596615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:35.596615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:14:35.596615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:14:35.596615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 92/188] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 30 18:14:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:35.596615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ffa9f-0a3c-4f79-bf60-7667c8219307 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:35.675001 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:35.685372 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 97/189] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:14:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:14:35.686335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c934cf4-4b38-402c-b71b-11d1db3fb6c8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:35.725475 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ffa9f-0a3c-4f79-bf60-7667c8219307 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:35.728113 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:35.770254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e263a6aa-1dfa-41ad-ac2d-2fbda610aee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:35.771056 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263a6aa-1dfa-41ad-ac2d-2fbda610aee7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:35.771348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263a6aa-1dfa-41ad-ac2d-2fbda610aee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:35.771628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263a6aa-1dfa-41ad-ac2d-2fbda610aee7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:35.772820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e263a6aa-1dfa-41ad-ac2d-2fbda610aee7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:35.775738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e263a6aa-1dfa-41ad-ac2d-2fbda610aee7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:35.777486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 93/190] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:14:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:35.778303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e263a6aa-1dfa-41ad-ac2d-2fbda610aee7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:35.808663 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c934cf4-4b38-402c-b71b-11d1db3fb6c8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:35.848207 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaab3ee-dce4-41f5-87ad-0654bf0c32f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:35.848887 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaab3ee-dce4-41f5-87ad-0654bf0c32f6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:35.849176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaab3ee-dce4-41f5-87ad-0654bf0c32f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:35.849456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaab3ee-dce4-41f5-87ad-0654bf0c32f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:35.850304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaab3ee-dce4-41f5-87ad-0654bf0c32f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:35.850618 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaab3ee-dce4-41f5-87ad-0654bf0c32f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:35.852561 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaab3ee-dce4-41f5-87ad-0654bf0c32f6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:35.861659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 98/191] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:35.862530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaab3ee-dce4-41f5-87ad-0654bf0c32f6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:35.910714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e263a6aa-1dfa-41ad-ac2d-2fbda610aee7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:35.932291 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:35.955582 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ddd8c1-f170-490a-b7f3-7ceaec2c134f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:35.956313 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddd8c1-f170-490a-b7f3-7ceaec2c134f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:35.956602 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddd8c1-f170-490a-b7f3-7ceaec2c134f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:35.956881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddd8c1-f170-490a-b7f3-7ceaec2c134f None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:35.957683 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ddd8c1-f170-490a-b7f3-7ceaec2c134f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:35.957996 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ddd8c1-f170-490a-b7f3-7ceaec2c134f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:35.959412 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ddd8c1-f170-490a-b7f3-7ceaec2c134f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:35.969084 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 94/192] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:14:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:35.969976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ddd8c1-f170-490a-b7f3-7ceaec2c134f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:35.990494 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaab3ee-dce4-41f5-87ad-0654bf0c32f6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:36.010032 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:36.031425 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50eecfd-398e-407c-9f4e-c598107c6b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:36.032275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50eecfd-398e-407c-9f4e-c598107c6b07 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:36.032763 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50eecfd-398e-407c-9f4e-c598107c6b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:36.033157 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50eecfd-398e-407c-9f4e-c598107c6b07 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:36.034307 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50eecfd-398e-407c-9f4e-c598107c6b07 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:36.034825 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50eecfd-398e-407c-9f4e-c598107c6b07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:36.036813 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50eecfd-398e-407c-9f4e-c598107c6b07 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:36.048951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 99/193] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:14:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:36.049896 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50eecfd-398e-407c-9f4e-c598107c6b07 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:36.117156 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ddd8c1-f170-490a-b7f3-7ceaec2c134f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:36.157477 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d91ad-618b-451f-994c-9eb4a2a34a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:36.158391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d91ad-618b-451f-994c-9eb4a2a34a85 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:36.158678 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d91ad-618b-451f-994c-9eb4a2a34a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:36.158961 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d91ad-618b-451f-994c-9eb4a2a34a85 None admin] PATH_INFO: `/v3/projects/65dabe170c1548f3831172ac12483e04` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:36.160294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88d91ad-618b-451f-994c-9eb4a2a34a85 None admin] RBAC: Authorizing `identity:get_project(project_id=65dabe170c1548f3831172ac12483e04)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:36.161820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88d91ad-618b-451f-994c-9eb4a2a34a85 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:36.164120 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 95/194] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:35 2022] GET /identity/v3/projects/65dabe170c1548f3831172ac12483e04 => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:36.165244 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d91ad-618b-451f-994c-9eb4a2a34a85 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:36.184189 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50eecfd-398e-407c-9f4e-c598107c6b07 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:36.199979 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:36.221760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec96a8-eb5e-410b-918b-af3baf79692c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:36.222401 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec96a8-eb5e-410b-918b-af3baf79692c None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:36.222659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec96a8-eb5e-410b-918b-af3baf79692c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:36.223361 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec96a8-eb5e-410b-918b-af3baf79692c None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:36.223642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec96a8-eb5e-410b-918b-af3baf79692c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:36.225142 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec96a8-eb5e-410b-918b-af3baf79692c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:36.246317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 100/195] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 30 18:14:36 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:14:36.247171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec96a8-eb5e-410b-918b-af3baf79692c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:36.294061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:36.298148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d91ad-618b-451f-994c-9eb4a2a34a85 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:36.348760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181e2b6-d08b-4505-b77c-4769b41ab9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:36.350278 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181e2b6-d08b-4505-b77c-4769b41ab9a4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:36.350783 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181e2b6-d08b-4505-b77c-4769b41ab9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:36.350947 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181e2b6-d08b-4505-b77c-4769b41ab9a4 None admin] PATH_INFO: `/v3/users/1d706385db2d4394aa1839ebfaaa37f5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:36.371618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181e2b6-d08b-4505-b77c-4769b41ab9a4 None admin] RBAC: Authorizing `identity:get_user(user_id=1d706385db2d4394aa1839ebfaaa37f5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:36.374025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181e2b6-d08b-4505-b77c-4769b41ab9a4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:36.376488 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 96/196] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:36 2022] GET /identity/v3/users/1d706385db2d4394aa1839ebfaaa37f5 => generated 283 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:36.383343 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181e2b6-d08b-4505-b77c-4769b41ab9a4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:36.396260 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec96a8-eb5e-410b-918b-af3baf79692c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:36.450425 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f34f4-6122-4a58-b132-5354a154dc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:36.451352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f34f4-6122-4a58-b132-5354a154dc53 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:36.451643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f34f4-6122-4a58-b132-5354a154dc53 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:36.451928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f34f4-6122-4a58-b132-5354a154dc53 None admin] PATH_INFO: `/v3/users/4ea1326eeb214c65856f79b22e345c8c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:36.453261 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4f34f4-6122-4a58-b132-5354a154dc53 None admin] RBAC: Authorizing `identity:get_user(user_id=4ea1326eeb214c65856f79b22e345c8c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:36.454834 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4f34f4-6122-4a58-b132-5354a154dc53 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:36.456646 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 101/197] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:36 2022] GET /identity/v3/users/4ea1326eeb214c65856f79b22e345c8c => generated 289 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:36.457399 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f34f4-6122-4a58-b132-5354a154dc53 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:36.519427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:36.527524 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181e2b6-d08b-4505-b77c-4769b41ab9a4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:36.584889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96882515-3099-43cc-ac19-f8ab18acfe21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:36.586353 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96882515-3099-43cc-ac19-f8ab18acfe21 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:36.586636 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96882515-3099-43cc-ac19-f8ab18acfe21 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:36.586919 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96882515-3099-43cc-ac19-f8ab18acfe21 None admin] PATH_INFO: `/v3/users/1d706385db2d4394aa1839ebfaaa37f5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:36.588432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96882515-3099-43cc-ac19-f8ab18acfe21 None admin] RBAC: Authorizing `identity:get_user(user_id=1d706385db2d4394aa1839ebfaaa37f5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:36.590070 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96882515-3099-43cc-ac19-f8ab18acfe21 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:36.591821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 97/198] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:36 2022] GET /identity/v3/users/1d706385db2d4394aa1839ebfaaa37f5 => generated 283 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:36.592636 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96882515-3099-43cc-ac19-f8ab18acfe21 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:36.621745 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f34f4-6122-4a58-b132-5354a154dc53 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:36.629769 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:36.674904 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c45ed0-c684-42d5-9a4d-2c90f547c591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:36.675581 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c45ed0-c684-42d5-9a4d-2c90f547c591 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:36.675871 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c45ed0-c684-42d5-9a4d-2c90f547c591 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:36.676140 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c45ed0-c684-42d5-9a4d-2c90f547c591 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:36.676988 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c45ed0-c684-42d5-9a4d-2c90f547c591 None admin] RBAC: Adding query filter params (role.id=3fbb71ae2de04e78b68d6d6f407ef09c, scope.project.id=65dabe170c1548f3831172ac12483e04, user.id=e13874941b204c6288bd9759b15662fb) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:36.677299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c45ed0-c684-42d5-9a4d-2c90f547c591 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:36.679036 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c45ed0-c684-42d5-9a4d-2c90f547c591 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:36.686341 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 102/199] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:14:36 2022] GET /identity/v3/role_assignments?user.id=e13874941b204c6288bd9759b15662fb&scope.project.id=65dabe170c1548f3831172ac12483e04&role.id=3fbb71ae2de04e78b68d6d6f407ef09c => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:36.687304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c45ed0-c684-42d5-9a4d-2c90f547c591 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:36.739006 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96882515-3099-43cc-ac19-f8ab18acfe21 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:36.788944 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c847c0c-64fe-408f-b4b0-f88adbabd2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:36.789990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c847c0c-64fe-408f-b4b0-f88adbabd2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:36.790555 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c847c0c-64fe-408f-b4b0-f88adbabd2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:36.790988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c847c0c-64fe-408f-b4b0-f88adbabd2a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:36.875175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c847c0c-64fe-408f-b4b0-f88adbabd2a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:36.877208 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c847c0c-64fe-408f-b4b0-f88adbabd2a2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:36.910335 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:36.913055 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c45ed0-c684-42d5-9a4d-2c90f547c591 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:36.916057 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:37.024874 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a72f3b-9e0b-457f-970b-b55a7e52da05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:37.024874 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a72f3b-9e0b-457f-970b-b55a7e52da05 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:37.024874 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a72f3b-9e0b-457f-970b-b55a7e52da05 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:37.026006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a72f3b-9e0b-457f-970b-b55a7e52da05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:37.029555 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 98/200] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:14:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:14:37.034342 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c847c0c-64fe-408f-b4b0-f88adbabd2a2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:37.121014 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a72f3b-9e0b-457f-970b-b55a7e52da05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:37.123061 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a72f3b-9e0b-457f-970b-b55a7e52da05 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:37.172552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c847c0c-64fe-408f-b4b0-f88adbabd2a2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:37.208588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:37.220852 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1208e0d-d9fe-4a4f-b0aa-1d94871b6a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:37.221660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1208e0d-d9fe-4a4f-b0aa-1d94871b6a7c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:37.222036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1208e0d-d9fe-4a4f-b0aa-1d94871b6a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:37.222389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1208e0d-d9fe-4a4f-b0aa-1d94871b6a7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:37.255980 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 103/201] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:14:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:14:37.257163 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a72f3b-9e0b-457f-970b-b55a7e52da05 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:37.260582 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:37.329688 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1208e0d-d9fe-4a4f-b0aa-1d94871b6a7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:37.331936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1208e0d-d9fe-4a4f-b0aa-1d94871b6a7c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:37.394988 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a72f3b-9e0b-457f-970b-b55a7e52da05 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:37.461511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554a03f-1993-4e53-8c36-b3e282b2a151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:37.462493 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554a03f-1993-4e53-8c36-b3e282b2a151 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:37.462780 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554a03f-1993-4e53-8c36-b3e282b2a151 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:37.463050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554a03f-1993-4e53-8c36-b3e282b2a151 None admin] PATH_INFO: `/v3/projects/9beede2985d54385bc4532fb19a265a5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:37.464477 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554a03f-1993-4e53-8c36-b3e282b2a151 None admin] RBAC: Authorizing `identity:get_project(project_id=9beede2985d54385bc4532fb19a265a5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:37.466117 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554a03f-1993-4e53-8c36-b3e282b2a151 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:37.468176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 104/202] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:37 2022] GET /identity/v3/projects/9beede2985d54385bc4532fb19a265a5 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:37.473945 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554a03f-1993-4e53-8c36-b3e282b2a151 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:37.487909 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 99/203] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:14:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:14:37.489216 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1208e0d-d9fe-4a4f-b0aa-1d94871b6a7c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:37.538282 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:37.584673 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:37.634376 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554a03f-1993-4e53-8c36-b3e282b2a151 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:37.669880 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1208e0d-d9fe-4a4f-b0aa-1d94871b6a7c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:37.681486 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4611c6f-cad1-48f8-ad90-43ba46adad3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:37.682548 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4611c6f-cad1-48f8-ad90-43ba46adad3b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:37.682843 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4611c6f-cad1-48f8-ad90-43ba46adad3b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:37.683136 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4611c6f-cad1-48f8-ad90-43ba46adad3b None admin] PATH_INFO: `/v3/projects/d5962edfa0c94d88a1056c1ed5e05058` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:37.684687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4611c6f-cad1-48f8-ad90-43ba46adad3b None admin] RBAC: Authorizing `identity:get_project(project_id=d5962edfa0c94d88a1056c1ed5e05058)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:37.686466 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4611c6f-cad1-48f8-ad90-43ba46adad3b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:37.688686 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 105/204] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:37 2022] GET /identity/v3/projects/d5962edfa0c94d88a1056c1ed5e05058 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:37.689530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4611c6f-cad1-48f8-ad90-43ba46adad3b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:37.718600 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5049922-65fb-48dc-8f9e-388a7b4a3d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:37.719265 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5049922-65fb-48dc-8f9e-388a7b4a3d54 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:37.719550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5049922-65fb-48dc-8f9e-388a7b4a3d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:37.719829 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5049922-65fb-48dc-8f9e-388a7b4a3d54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:37.720679 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5049922-65fb-48dc-8f9e-388a7b4a3d54 None admin] RBAC: Adding query filter params (role.id=0ef3687af20a443281e3cfdceca3597d, scope.project.id=9beede2985d54385bc4532fb19a265a5, user.id=1d706385db2d4394aa1839ebfaaa37f5) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:37.720992 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5049922-65fb-48dc-8f9e-388a7b4a3d54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:37.722528 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5049922-65fb-48dc-8f9e-388a7b4a3d54 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:37.729720 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 100/205] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:14:37 2022] GET /identity/v3/role_assignments?user.id=1d706385db2d4394aa1839ebfaaa37f5&scope.project.id=9beede2985d54385bc4532fb19a265a5&role.id=0ef3687af20a443281e3cfdceca3597d => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:37.738716 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5049922-65fb-48dc-8f9e-388a7b4a3d54 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:37.845864 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4611c6f-cad1-48f8-ad90-43ba46adad3b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:37.904093 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3aa641-9b3f-4054-8391-68e06f6befcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:37.904979 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3aa641-9b3f-4054-8391-68e06f6befcc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:37.905262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3aa641-9b3f-4054-8391-68e06f6befcc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:37.905541 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3aa641-9b3f-4054-8391-68e06f6befcc None admin] PATH_INFO: `/v3/projects/9beede2985d54385bc4532fb19a265a5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:37.906876 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3aa641-9b3f-4054-8391-68e06f6befcc None admin] RBAC: Authorizing `identity:get_project(project_id=9beede2985d54385bc4532fb19a265a5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:37.908485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3aa641-9b3f-4054-8391-68e06f6befcc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:37.910827 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 106/206] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:37 2022] GET /identity/v3/projects/9beede2985d54385bc4532fb19a265a5 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:37.920021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3aa641-9b3f-4054-8391-68e06f6befcc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:37.930435 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:37.936478 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:37.944634 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5049922-65fb-48dc-8f9e-388a7b4a3d54 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:37.995764 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c726f7-fdc2-4260-9004-a295670e07a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:37.997356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c726f7-fdc2-4260-9004-a295670e07a1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:37.998013 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c726f7-fdc2-4260-9004-a295670e07a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:37.998670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c726f7-fdc2-4260-9004-a295670e07a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:38.000012 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c726f7-fdc2-4260-9004-a295670e07a1 None admin] RBAC: Adding query filter params (role.id=bdbac30ca21b4146bf95b3a053dff30a, scope.project.id=d5962edfa0c94d88a1056c1ed5e05058, user.id=4ea1326eeb214c65856f79b22e345c8c) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:38.000600 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c726f7-fdc2-4260-9004-a295670e07a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:38.002310 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c726f7-fdc2-4260-9004-a295670e07a1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:38.017654 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 101/207] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:14:37 2022] GET /identity/v3/role_assignments?user.id=4ea1326eeb214c65856f79b22e345c8c&scope.project.id=d5962edfa0c94d88a1056c1ed5e05058&role.id=bdbac30ca21b4146bf95b3a053dff30a => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:38.054801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3aa641-9b3f-4054-8391-68e06f6befcc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:38.099695 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c52fc-2b27-4b28-907a-de9aa7bd5763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:38.100285 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c52fc-2b27-4b28-907a-de9aa7bd5763 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:38.100567 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c52fc-2b27-4b28-907a-de9aa7bd5763 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:38.102949 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c52fc-2b27-4b28-907a-de9aa7bd5763 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:38.102949 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691c52fc-2b27-4b28-907a-de9aa7bd5763 None admin] RBAC: Adding query filter params (role.id=d5f874a3d78f4ee0ade318e1bbebf7ec, scope.project.id=9beede2985d54385bc4532fb19a265a5, user.id=1d706385db2d4394aa1839ebfaaa37f5) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:38.102949 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691c52fc-2b27-4b28-907a-de9aa7bd5763 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:38.103975 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691c52fc-2b27-4b28-907a-de9aa7bd5763 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:38.110846 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 107/208] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:14:37 2022] GET /identity/v3/role_assignments?user.id=1d706385db2d4394aa1839ebfaaa37f5&scope.project.id=9beede2985d54385bc4532fb19a265a5&role.id=d5f874a3d78f4ee0ade318e1bbebf7ec => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:38.717147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a2cb2-4250-46df-9655-80c9d5676229 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:38.717696 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a2cb2-4250-46df-9655-80c9d5676229 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:38.717804 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a2cb2-4250-46df-9655-80c9d5676229 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:38.719266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 102/209] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:38.724999 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fbc0a-72bb-4900-8c09-adfab56fbd55 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:38.725410 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fbc0a-72bb-4900-8c09-adfab56fbd55 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:38.725724 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fbc0a-72bb-4900-8c09-adfab56fbd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:38.744137 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-479fbc0a-72bb-4900-8c09-adfab56fbd55 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:38.745361 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-479fbc0a-72bb-4900-8c09-adfab56fbd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:38.830729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:38.836571 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 108/210] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:38.860343 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a2cb2-4250-46df-9655-80c9d5676229 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:39.044491 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a2cb2-4250-46df-9655-80c9d5676229 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:39.095623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293a3b3-982c-4f13-a9f8-bf75555300b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:39.096467 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293a3b3-982c-4f13-a9f8-bf75555300b2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:39.096826 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293a3b3-982c-4f13-a9f8-bf75555300b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:39.097103 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293a3b3-982c-4f13-a9f8-bf75555300b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:39.109979 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293a3b3-982c-4f13-a9f8-bf75555300b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:39.119072 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-7293a3b3-982c-4f13-a9f8-bf75555300b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:14:39.119072 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:39.119072 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:39.119072 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:39.119072 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:39.119072 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:39.119072 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:39.119072 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:39.119072 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:39.119072 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:39.119072 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:39.119072 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:39.119072 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:39.119072 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:39.119072 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:39.119072 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:39.120799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:39.120799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:39.120799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:39.120799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:39.120799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:39.120799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:39.120799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:39.120799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:39.120799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:39.120799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:39.120799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:39.120799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:39.120799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:39.120799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:39.120799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:39.120799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:39.122433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:39.122433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:39.122433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:39.122433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:39.122433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:39.122433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:39.122433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:39.122433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:39.122433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:39.122433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:39.122433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:14:39.122433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:14:39.122433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 103/211] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:39.131862 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fbc0a-72bb-4900-8c09-adfab56fbd55 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:39.158402 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b79a9f-62ae-4e47-9ae3-568a40e55e77 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:39.158753 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b79a9f-62ae-4e47-9ae3-568a40e55e77 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:39.158949 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b79a9f-62ae-4e47-9ae3-568a40e55e77 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:39.160352 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 104/212] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:14:39.166869 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f4228-9903-4691-9b9c-7f7924f6cbdf None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:39.167378 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f4228-9903-4691-9b9c-7f7924f6cbdf None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:39.167831 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f4228-9903-4691-9b9c-7f7924f6cbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:39.189140 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8f7f4228-9903-4691-9b9c-7f7924f6cbdf None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:39.190746 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8f7f4228-9903-4691-9b9c-7f7924f6cbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:39.202892 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:39.278846 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fbc0a-72bb-4900-8c09-adfab56fbd55 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:39.316773 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 105/213] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 18:14:39.330584 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb915f-fbaa-41fd-a0d2-35f7eb65f6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:39.332079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb915f-fbaa-41fd-a0d2-35f7eb65f6dd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:39.332789 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb915f-fbaa-41fd-a0d2-35f7eb65f6dd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:39.333491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb915f-fbaa-41fd-a0d2-35f7eb65f6dd None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:39.335181 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb915f-fbaa-41fd-a0d2-35f7eb65f6dd None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:39.336643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb915f-fbaa-41fd-a0d2-35f7eb65f6dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:39.339554 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb915f-fbaa-41fd-a0d2-35f7eb65f6dd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:39.343037 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f4228-9903-4691-9b9c-7f7924f6cbdf None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:39.361540 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 109/214] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:14:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:39.371047 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb915f-fbaa-41fd-a0d2-35f7eb65f6dd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:39.468730 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f4228-9903-4691-9b9c-7f7924f6cbdf None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:39.497615 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:39.520863 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c52733-c2f8-46bd-9652-2da61beec29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:39.522115 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c52733-c2f8-46bd-9652-2da61beec29a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:39.522638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c52733-c2f8-46bd-9652-2da61beec29a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:39.523152 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c52733-c2f8-46bd-9652-2da61beec29a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:39.524734 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c52733-c2f8-46bd-9652-2da61beec29a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:39.526519 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c52733-c2f8-46bd-9652-2da61beec29a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:39.528455 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 106/215] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:14:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:39.534320 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c52733-c2f8-46bd-9652-2da61beec29a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:39.551990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:39.562157 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb915f-fbaa-41fd-a0d2-35f7eb65f6dd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:39.600291 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee753d1f-fc97-4e56-b4e4-91cf647c3780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:39.601524 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee753d1f-fc97-4e56-b4e4-91cf647c3780 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:39.601910 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee753d1f-fc97-4e56-b4e4-91cf647c3780 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:39.602196 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee753d1f-fc97-4e56-b4e4-91cf647c3780 None admin] PATH_INFO: `/v3/users/96fabca16871409d86be75dcc35b51cc` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:39.619289 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee753d1f-fc97-4e56-b4e4-91cf647c3780 None admin] RBAC: Authorizing `identity:get_user(user_id=96fabca16871409d86be75dcc35b51cc)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:39.620324 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee753d1f-fc97-4e56-b4e4-91cf647c3780 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:39.622161 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 110/216] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:39 2022] GET /identity/v3/users/96fabca16871409d86be75dcc35b51cc => generated 297 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:39.627777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee753d1f-fc97-4e56-b4e4-91cf647c3780 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:39.660152 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c52733-c2f8-46bd-9652-2da61beec29a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:39.703470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09542a2-6db0-4f9e-b47e-eeec1d47b096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:39.704675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09542a2-6db0-4f9e-b47e-eeec1d47b096 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:39.705167 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09542a2-6db0-4f9e-b47e-eeec1d47b096 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:39.705850 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09542a2-6db0-4f9e-b47e-eeec1d47b096 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:39.706799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09542a2-6db0-4f9e-b47e-eeec1d47b096 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:39.708497 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09542a2-6db0-4f9e-b47e-eeec1d47b096 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:39.741855 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 107/217] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 30 18:14:39 2022] POST /identity/v3/users => generated 297 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:14:39.744507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee753d1f-fc97-4e56-b4e4-91cf647c3780 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:39.781073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:39.816985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea503b1-b0c9-4893-afe8-8e21a6310796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:39.818096 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea503b1-b0c9-4893-afe8-8e21a6310796 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:39.818666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea503b1-b0c9-4893-afe8-8e21a6310796 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:39.819179 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea503b1-b0c9-4893-afe8-8e21a6310796 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:39.927826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea503b1-b0c9-4893-afe8-8e21a6310796 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:39.928309 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea503b1-b0c9-4893-afe8-8e21a6310796 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:40.085013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 111/218] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:14:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 18:14:40.092625 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09542a2-6db0-4f9e-b47e-eeec1d47b096 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:40.146455 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee16d06-cc0a-47c1-8bdc-42aa151cf0ae None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:40.147085 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee16d06-cc0a-47c1-8bdc-42aa151cf0ae None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:40.147085 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee16d06-cc0a-47c1-8bdc-42aa151cf0ae None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:40.149500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 112/219] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:40.155227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cff36-0f8d-41ba-a604-4d8ebdf9b8c2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:40.155749 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cff36-0f8d-41ba-a604-4d8ebdf9b8c2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:40.156171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cff36-0f8d-41ba-a604-4d8ebdf9b8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:40.192510 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-838cff36-0f8d-41ba-a604-4d8ebdf9b8c2 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:40.195604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-838cff36-0f8d-41ba-a604-4d8ebdf9b8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:40.226359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:40.253528 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:40.304340 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 113/220] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:40.325281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09542a2-6db0-4f9e-b47e-eeec1d47b096 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:40.336722 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838cff36-0f8d-41ba-a604-4d8ebdf9b8c2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:40.427992 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56759e88-d0ce-48ba-bb5a-17debeb506b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:40.432166 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56759e88-d0ce-48ba-bb5a-17debeb506b6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:40.432923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56759e88-d0ce-48ba-bb5a-17debeb506b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:40.434130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56759e88-d0ce-48ba-bb5a-17debeb506b6 None admin] PATH_INFO: `/v3/projects/9beede2985d54385bc4532fb19a265a5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:40.442073 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56759e88-d0ce-48ba-bb5a-17debeb506b6 None admin] RBAC: Authorizing `identity:get_project(project_id=9beede2985d54385bc4532fb19a265a5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:40.447950 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56759e88-d0ce-48ba-bb5a-17debeb506b6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:40.453610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 108/221] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:40 2022] GET /identity/v3/projects/9beede2985d54385bc4532fb19a265a5 => generated 302 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:40.466812 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56759e88-d0ce-48ba-bb5a-17debeb506b6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:40.609218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838cff36-0f8d-41ba-a604-4d8ebdf9b8c2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:40.682819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846299b2-5a37-4ca5-8684-f3cbc7677613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:40.684751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846299b2-5a37-4ca5-8684-f3cbc7677613 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:40.685515 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846299b2-5a37-4ca5-8684-f3cbc7677613 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:40.686279 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846299b2-5a37-4ca5-8684-f3cbc7677613 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:40.689118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56759e88-d0ce-48ba-bb5a-17debeb506b6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:40.707335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846299b2-5a37-4ca5-8684-f3cbc7677613 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:40.724650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-846299b2-5a37-4ca5-8684-f3cbc7677613 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:14:40.724650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:40.724650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:40.724650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:40.724650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:40.724650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:40.724650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:40.724650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:40.724650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:40.724650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:40.724650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:40.724650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:40.724650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:40.724650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:40.724650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:40.724650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:40.727234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:40.727234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:40.727234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:40.727234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:40.727234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:40.727234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:40.727234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:40.727234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:40.727234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:40.727234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:40.727234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:40.727234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:40.727234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:40.727234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:40.727234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:40.727234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:40.729068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:40.729068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:40.729068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:40.729068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:40.729068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:40.729068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:40.729068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:40.729068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:40.729068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:40.729068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:40.729068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:14:40.729068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:14:40.733747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 114/222] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:14:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:14:40.736463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b5401-6836-428a-99ee-3862aad7aa5c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:40.737151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b5401-6836-428a-99ee-3862aad7aa5c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:40.737830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b5401-6836-428a-99ee-3862aad7aa5c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:40.741923 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 115/223] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:40.746976 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b5401-6836-428a-99ee-3862aad7aa5c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:40.762994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:40.768659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97586e79-dee8-4f42-aa36-88f06c812746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:40.770638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97586e79-dee8-4f42-aa36-88f06c812746 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:40.771389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97586e79-dee8-4f42-aa36-88f06c812746 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:40.771958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97586e79-dee8-4f42-aa36-88f06c812746 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:40.774063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97586e79-dee8-4f42-aa36-88f06c812746 None admin] RBAC: Adding query filter params (role.id=3fbb71ae2de04e78b68d6d6f407ef09c, scope.project.id=9beede2985d54385bc4532fb19a265a5, user.id=96fabca16871409d86be75dcc35b51cc) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:40.775185 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97586e79-dee8-4f42-aa36-88f06c812746 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:40.778868 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97586e79-dee8-4f42-aa36-88f06c812746 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:40.785517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:40.797777 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 109/224] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:14:40 2022] GET /identity/v3/role_assignments?user.id=96fabca16871409d86be75dcc35b51cc&scope.project.id=9beede2985d54385bc4532fb19a265a5&role.id=3fbb71ae2de04e78b68d6d6f407ef09c => generated 259 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:40.799548 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0e98d-3473-4897-b29c-f94010b257cb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:40.800883 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0e98d-3473-4897-b29c-f94010b257cb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:40.801643 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0e98d-3473-4897-b29c-f94010b257cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:40.832750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0ac0e98d-3473-4897-b29c-f94010b257cb None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:40.834526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0ac0e98d-3473-4897-b29c-f94010b257cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:40.976107 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b5401-6836-428a-99ee-3862aad7aa5c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:41.008157 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 110/225] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:41.044884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0e98d-3473-4897-b29c-f94010b257cb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:41.052813 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866ccf8-7402-4379-ae97-7be0c0f6cc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:41.054556 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866ccf8-7402-4379-ae97-7be0c0f6cc2d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:41.055086 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866ccf8-7402-4379-ae97-7be0c0f6cc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:41.057073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866ccf8-7402-4379-ae97-7be0c0f6cc2d None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:41.058586 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1866ccf8-7402-4379-ae97-7be0c0f6cc2d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:41.059130 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1866ccf8-7402-4379-ae97-7be0c0f6cc2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:41.064761 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1866ccf8-7402-4379-ae97-7be0c0f6cc2d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:41.085188 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 116/226] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:41.096854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866ccf8-7402-4379-ae97-7be0c0f6cc2d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:41.262749 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0e98d-3473-4897-b29c-f94010b257cb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:41.266034 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:41.332476 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866ccf8-7402-4379-ae97-7be0c0f6cc2d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:41.359564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:41.402879 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f563132-2013-4421-8b9c-b460ee67516a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:41.404416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f563132-2013-4421-8b9c-b460ee67516a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:41.404910 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f563132-2013-4421-8b9c-b460ee67516a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:41.405566 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f563132-2013-4421-8b9c-b460ee67516a None admin] PATH_INFO: `/v3/users/1d706385db2d4394aa1839ebfaaa37f5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:41.406009 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ecf6a-afc7-4e0f-9845-8562f1c2d013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:41.407018 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ecf6a-afc7-4e0f-9845-8562f1c2d013 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:41.407307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ecf6a-afc7-4e0f-9845-8562f1c2d013 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:41.407583 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ecf6a-afc7-4e0f-9845-8562f1c2d013 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:41.407916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f563132-2013-4421-8b9c-b460ee67516a None admin] RBAC: Authorizing `identity:get_user(user_id=1d706385db2d4394aa1839ebfaaa37f5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:41.411077 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f563132-2013-4421-8b9c-b460ee67516a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:41.415446 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 117/227] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:41 2022] GET /identity/v3/users/1d706385db2d4394aa1839ebfaaa37f5 => generated 283 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:41.425673 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f563132-2013-4421-8b9c-b460ee67516a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:41.462298 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ecf6a-afc7-4e0f-9845-8562f1c2d013 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:41.470706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-305ecf6a-afc7-4e0f-9845-8562f1c2d013 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:14:41.470706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:41.470706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:41.470706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:41.470706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:41.470706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:41.470706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:41.470706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:41.470706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:41.470706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:41.470706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:41.470706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:41.470706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:41.470706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:41.470706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:41.470706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:41.471685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:41.471685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:41.471685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:41.471685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:41.471685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:41.471685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:41.471685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:41.471685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:41.471685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:41.471685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:41.471685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:41.471685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:41.471685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:41.471685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:41.471685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:41.471685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:41.472465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:41.472465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:41.472465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:41.472465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:41.472465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:41.472465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:41.472465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:41.472465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:41.472465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:41.472465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:41.472465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:14:41.472465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:14:41.477869 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 111/228] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 30 18:14:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:14:41.482624 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ecf6a-afc7-4e0f-9845-8562f1c2d013 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:41.597615 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f563132-2013-4421-8b9c-b460ee67516a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:41.640352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a65b450-2db6-466e-b5d4-e5a071134d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:41.641247 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65b450-2db6-466e-b5d4-e5a071134d28 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:41.641533 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65b450-2db6-466e-b5d4-e5a071134d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:41.641851 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65b450-2db6-466e-b5d4-e5a071134d28 None admin] PATH_INFO: `/v3/projects/9beede2985d54385bc4532fb19a265a5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:41.643832 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a65b450-2db6-466e-b5d4-e5a071134d28 None admin] RBAC: Authorizing `identity:get_project(project_id=9beede2985d54385bc4532fb19a265a5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:41.645633 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a65b450-2db6-466e-b5d4-e5a071134d28 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:41.648004 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 118/229] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:41 2022] GET /identity/v3/projects/9beede2985d54385bc4532fb19a265a5 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:41.656842 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a65b450-2db6-466e-b5d4-e5a071134d28 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:41.674746 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:41.720286 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ecf6a-afc7-4e0f-9845-8562f1c2d013 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:41.800452 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c228c3-d01c-4565-8f39-d1f0f3066769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:41.801221 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c228c3-d01c-4565-8f39-d1f0f3066769 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:41.801434 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c228c3-d01c-4565-8f39-d1f0f3066769 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:41.801721 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c228c3-d01c-4565-8f39-d1f0f3066769 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:41.802499 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c228c3-d01c-4565-8f39-d1f0f3066769 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:41.802810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c228c3-d01c-4565-8f39-d1f0f3066769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:41.804796 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c228c3-d01c-4565-8f39-d1f0f3066769 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:41.827731 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 112/230] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:14:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:41.836924 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c228c3-d01c-4565-8f39-d1f0f3066769 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:41.904413 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a65b450-2db6-466e-b5d4-e5a071134d28 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:41.949752 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:42.020695 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c228c3-d01c-4565-8f39-d1f0f3066769 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:42.043515 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c800ec-b04e-4afd-a2ae-684451e231c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:42.045823 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c800ec-b04e-4afd-a2ae-684451e231c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:42.046685 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c800ec-b04e-4afd-a2ae-684451e231c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:42.047626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c800ec-b04e-4afd-a2ae-684451e231c0 None admin] PATH_INFO: `/v3/projects/9beede2985d54385bc4532fb19a265a5/users/1d706385db2d4394aa1839ebfaaa37f5/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:42.057618 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c800ec-b04e-4afd-a2ae-684451e231c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9beede2985d54385bc4532fb19a265a5, user_id=1d706385db2d4394aa1839ebfaaa37f5, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:42.073908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c800ec-b04e-4afd-a2ae-684451e231c0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:42.096735 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151211d-d0db-4bb0-a7bd-c98103c71a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:42.101720 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 119/231] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:14:41 2022] PUT /identity/v3/projects/9beede2985d54385bc4532fb19a265a5/users/1d706385db2d4394aa1839ebfaaa37f5/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:14:42.102396 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151211d-d0db-4bb0-a7bd-c98103c71a37 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:42.102691 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151211d-d0db-4bb0-a7bd-c98103c71a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:42.102978 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151211d-d0db-4bb0-a7bd-c98103c71a37 None admin] PATH_INFO: `/v3/users/1d706385db2d4394aa1839ebfaaa37f5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:42.104441 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1151211d-d0db-4bb0-a7bd-c98103c71a37 None admin] RBAC: Authorizing `identity:get_user(user_id=1d706385db2d4394aa1839ebfaaa37f5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:42.106038 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1151211d-d0db-4bb0-a7bd-c98103c71a37 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:42.108153 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 113/232] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:41 2022] GET /identity/v3/users/1d706385db2d4394aa1839ebfaaa37f5 => generated 283 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:42.116216 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c800ec-b04e-4afd-a2ae-684451e231c0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:42.214460 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:42.261092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c800ec-b04e-4afd-a2ae-684451e231c0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:42.311527 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553455dc-3741-49c3-b49e-0eb05fc00fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:42.312794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553455dc-3741-49c3-b49e-0eb05fc00fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:42.313078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553455dc-3741-49c3-b49e-0eb05fc00fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:42.313412 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553455dc-3741-49c3-b49e-0eb05fc00fe6 None admin] PATH_INFO: `/v3/projects/9beede2985d54385bc4532fb19a265a5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:42.315003 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553455dc-3741-49c3-b49e-0eb05fc00fe6 None admin] RBAC: Authorizing `identity:get_project(project_id=9beede2985d54385bc4532fb19a265a5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:42.316517 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553455dc-3741-49c3-b49e-0eb05fc00fe6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:42.318413 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 120/233] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:42 2022] GET /identity/v3/projects/9beede2985d54385bc4532fb19a265a5 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:42.326734 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151211d-d0db-4bb0-a7bd-c98103c71a37 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:42.474059 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151211d-d0db-4bb0-a7bd-c98103c71a37 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:42.475878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567364d-6b5e-46d1-a795-9b34e45e6d92 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:42.476789 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567364d-6b5e-46d1-a795-9b34e45e6d92 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:42.477117 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567364d-6b5e-46d1-a795-9b34e45e6d92 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:42.478652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 121/234] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:42.487211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294b1dd-f83b-485b-9ec8-d33eeb429e71 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:42.487490 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294b1dd-f83b-485b-9ec8-d33eeb429e71 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:42.487769 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294b1dd-f83b-485b-9ec8-d33eeb429e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:42.506730 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9294b1dd-f83b-485b-9ec8-d33eeb429e71 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:42.508488 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9294b1dd-f83b-485b-9ec8-d33eeb429e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:42.517081 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:42.553012 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc6cf0-8763-4e4e-9c21-c48075a35655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:42.554897 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc6cf0-8763-4e4e-9c21-c48075a35655 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:42.555389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc6cf0-8763-4e4e-9c21-c48075a35655 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:42.555881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc6cf0-8763-4e4e-9c21-c48075a35655 None admin] PATH_INFO: `/v3/projects/9beede2985d54385bc4532fb19a265a5/users/1d706385db2d4394aa1839ebfaaa37f5/roles/d5f874a3d78f4ee0ade318e1bbebf7ec` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:42.565368 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc6cf0-8763-4e4e-9c21-c48075a35655 None admin] RBAC: Authorizing `identity:create_grant(project_id=9beede2985d54385bc4532fb19a265a5, user_id=1d706385db2d4394aa1839ebfaaa37f5, role_id=d5f874a3d78f4ee0ade318e1bbebf7ec)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:42.568276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc6cf0-8763-4e4e-9c21-c48075a35655 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:42.603016 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 122/235] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:42.607192 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 114/236] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:14:42 2022] PUT /identity/v3/projects/9beede2985d54385bc4532fb19a265a5/users/1d706385db2d4394aa1839ebfaaa37f5/roles/d5f874a3d78f4ee0ade318e1bbebf7ec => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:14:42.627857 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294b1dd-f83b-485b-9ec8-d33eeb429e71 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:42.719341 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:42.772676 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294b1dd-f83b-485b-9ec8-d33eeb429e71 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:42.826558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166fa2b-8fca-49fa-b19e-6baab1923d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:42.827777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166fa2b-8fca-49fa-b19e-6baab1923d8b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:42.828282 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166fa2b-8fca-49fa-b19e-6baab1923d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:42.828795 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166fa2b-8fca-49fa-b19e-6baab1923d8b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:42.845467 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b166fa2b-8fca-49fa-b19e-6baab1923d8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:42.856283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b166fa2b-8fca-49fa-b19e-6baab1923d8b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 18:14:42.856283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:42.856283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:42.856283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:42.856283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:42.856283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:42.856283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:42.856283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:42.856283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:42.856283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:42.856283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:42.856283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:42.856283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:42.856283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:42.856283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:42.856283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:42.857236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:42.857236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:42.857236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:42.857236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:42.857236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:42.857236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:42.857236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:42.857236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:42.857236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:42.857236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:42.857236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:42.857236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:42.857236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:42.857236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:42.857236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:42.857236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:42.858059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:42.858059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:42.858059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:42.858059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:42.858059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:42.858059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:42.858059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:42.858059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:42.858059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:42.858059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:42.858059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 18:14:42.858059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:14:42.860607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 123/237] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:42.867580 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc6cf0-8763-4e4e-9c21-c48075a35655 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:43.036975 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc6cf0-8763-4e4e-9c21-c48075a35655 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:43.092864 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28812a93-5a77-4b0c-9afe-02d2d6adc7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:43.093984 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28812a93-5a77-4b0c-9afe-02d2d6adc7cd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:43.094481 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28812a93-5a77-4b0c-9afe-02d2d6adc7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:43.094984 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28812a93-5a77-4b0c-9afe-02d2d6adc7cd None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:43.096014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28812a93-5a77-4b0c-9afe-02d2d6adc7cd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:43.096536 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28812a93-5a77-4b0c-9afe-02d2d6adc7cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:43.098214 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28812a93-5a77-4b0c-9afe-02d2d6adc7cd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:43.121347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 115/238] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:14:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:43.126821 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166fa2b-8fca-49fa-b19e-6baab1923d8b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:43.196013 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20848149-3136-4ef3-a754-e2e2fa45b146 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:43.196548 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20848149-3136-4ef3-a754-e2e2fa45b146 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:43.197048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20848149-3136-4ef3-a754-e2e2fa45b146 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:43.200172 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 116/239] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:43.212126 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6575f-8a12-42b9-99a9-529e5e8699f8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:43.212621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6575f-8a12-42b9-99a9-529e5e8699f8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:43.213137 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6575f-8a12-42b9-99a9-529e5e8699f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:43.258369 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f9e6575f-8a12-42b9-99a9-529e5e8699f8 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:43.259760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f9e6575f-8a12-42b9-99a9-529e5e8699f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:43.331171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166fa2b-8fca-49fa-b19e-6baab1923d8b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:43.369641 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:43.415378 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:43.437278 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 117/240] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:43.463063 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1f917-b60b-49ba-bc34-e4e8493c72cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:43.463996 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1f917-b60b-49ba-bc34-e4e8493c72cb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:43.464276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1f917-b60b-49ba-bc34-e4e8493c72cb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:43.464543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1f917-b60b-49ba-bc34-e4e8493c72cb None admin] PATH_INFO: `/v3/users/e98c10b29c0d4866b532931fd1855e0c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:43.489315 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1f917-b60b-49ba-bc34-e4e8493c72cb None admin] RBAC: Authorizing `identity:get_user(user_id=e98c10b29c0d4866b532931fd1855e0c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:43.492330 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1f917-b60b-49ba-bc34-e4e8493c72cb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:43.494562 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6575f-8a12-42b9-99a9-529e5e8699f8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:43.495075 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 124/241] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:43 2022] GET /identity/v3/users/e98c10b29c0d4866b532931fd1855e0c => generated 297 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:43.505631 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1f917-b60b-49ba-bc34-e4e8493c72cb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:43.653028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6575f-8a12-42b9-99a9-529e5e8699f8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:43.693340 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d6efe-7f50-47d6-a365-6b04a71e1ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:43.694566 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d6efe-7f50-47d6-a365-6b04a71e1ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:43.694566 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d6efe-7f50-47d6-a365-6b04a71e1ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:43.695009 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d6efe-7f50-47d6-a365-6b04a71e1ea6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:43.700402 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d6efe-7f50-47d6-a365-6b04a71e1ea6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:43.700402 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d6efe-7f50-47d6-a365-6b04a71e1ea6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:43.700402 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 118/242] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:14:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:43.705015 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d6efe-7f50-47d6-a365-6b04a71e1ea6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:43.784139 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:43.817308 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1f917-b60b-49ba-bc34-e4e8493c72cb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:43.870135 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b39de2-f640-4223-9574-e18b760734e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:43.871091 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b39de2-f640-4223-9574-e18b760734e0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:43.871469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b39de2-f640-4223-9574-e18b760734e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:43.871752 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b39de2-f640-4223-9574-e18b760734e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:43.906088 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d6efe-7f50-47d6-a365-6b04a71e1ea6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:43.944341 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:43.953431 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247038ac-3c4b-4e5f-b8a2-37ab07334c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:43.954168 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247038ac-3c4b-4e5f-b8a2-37ab07334c76 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:43.954458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247038ac-3c4b-4e5f-b8a2-37ab07334c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:43.954739 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247038ac-3c4b-4e5f-b8a2-37ab07334c76 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:43.955477 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247038ac-3c4b-4e5f-b8a2-37ab07334c76 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:43.957080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247038ac-3c4b-4e5f-b8a2-37ab07334c76 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:43.966669 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b39de2-f640-4223-9574-e18b760734e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:43.968913 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b39de2-f640-4223-9574-e18b760734e0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:43.981945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 119/243] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 30 18:14:43 2022] POST /identity/v3/users => generated 293 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:14:43.983495 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144b505-ab11-4010-a3be-057b4159c140 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:43.984299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144b505-ab11-4010-a3be-057b4159c140 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:43.984299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144b505-ab11-4010-a3be-057b4159c140 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:43.986021 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 120/244] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:43.992791 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13e7e1-6be6-456a-a7c1-90f8ce23907c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:43.993264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13e7e1-6be6-456a-a7c1-90f8ce23907c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:43.993545 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13e7e1-6be6-456a-a7c1-90f8ce23907c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:44.042741 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ae13e7e1-6be6-456a-a7c1-90f8ce23907c None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:44.044276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ae13e7e1-6be6-456a-a7c1-90f8ce23907c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:44.109733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 125/245] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:14:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 18:14:44.125598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b39de2-f640-4223-9574-e18b760734e0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:44.283845 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 121/246] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:44.295274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:44.312671 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b39de2-f640-4223-9574-e18b760734e0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:44.337813 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13e7e1-6be6-456a-a7c1-90f8ce23907c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:44.357082 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c98008-69fe-48f3-b74e-5ea9634b49fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:44.358667 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c98008-69fe-48f3-b74e-5ea9634b49fb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:44.359549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c98008-69fe-48f3-b74e-5ea9634b49fb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:44.360044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c98008-69fe-48f3-b74e-5ea9634b49fb None admin] PATH_INFO: `/v3/projects/9beede2985d54385bc4532fb19a265a5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:44.361603 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c98008-69fe-48f3-b74e-5ea9634b49fb None admin] RBAC: Authorizing `identity:get_project(project_id=9beede2985d54385bc4532fb19a265a5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:44.363287 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c98008-69fe-48f3-b74e-5ea9634b49fb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:44.365288 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 126/247] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:44 2022] GET /identity/v3/projects/9beede2985d54385bc4532fb19a265a5 => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:44.371544 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c98008-69fe-48f3-b74e-5ea9634b49fb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:44.377157 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:44.527634 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13e7e1-6be6-456a-a7c1-90f8ce23907c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:44.557165 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c98008-69fe-48f3-b74e-5ea9634b49fb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:44.583964 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ca6d2-89af-47a2-b3f4-a9a2c6100eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:44.584782 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ca6d2-89af-47a2-b3f4-a9a2c6100eef None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:44.585643 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ca6d2-89af-47a2-b3f4-a9a2c6100eef None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:44.585959 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ca6d2-89af-47a2-b3f4-a9a2c6100eef None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:44.606473 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ca6d2-89af-47a2-b3f4-a9a2c6100eef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:44.612586 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75fda0-a5c6-4c3c-9a8f-b39b852160ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:44.613506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75fda0-a5c6-4c3c-9a8f-b39b852160ae None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:44.613972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75fda0-a5c6-4c3c-9a8f-b39b852160ae None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:44.614400 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75fda0-a5c6-4c3c-9a8f-b39b852160ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:44.614584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ae4ca6d2-89af-47a2-b3f4-a9a2c6100eef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:14:44.614584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:44.614584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:44.614584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:44.614584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:44.614584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:44.614584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:44.614584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:44.614584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:44.614584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:44.614584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:44.614584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:44.614584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:44.614584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:44.614584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:44.614584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:44.615714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:44.615714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:44.615714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:44.615714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:44.615714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:44.615714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:44.615714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:44.615714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:44.615714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:44.615714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:44.615714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:44.615714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:44.615714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:44.615714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:44.615714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:44.615714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:44.618148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:44.618148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:44.618148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:44.618148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:44.618148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:44.618148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:44.618148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:44.618148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:44.618148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:44.618148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:44.618148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:14:44.618148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:14:44.618148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 122/248] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:14:44.619822 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75fda0-a5c6-4c3c-9a8f-b39b852160ae None admin] RBAC: Adding query filter params (role.id=bdbac30ca21b4146bf95b3a053dff30a, scope.project.id=9beede2985d54385bc4532fb19a265a5, user.id=e98c10b29c0d4866b532931fd1855e0c) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:44.619822 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75fda0-a5c6-4c3c-9a8f-b39b852160ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:44.620482 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75fda0-a5c6-4c3c-9a8f-b39b852160ae None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:44.623157 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ca6d2-89af-47a2-b3f4-a9a2c6100eef None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:44.633917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 127/249] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:14:44 2022] GET /identity/v3/role_assignments?user.id=e98c10b29c0d4866b532931fd1855e0c&scope.project.id=9beede2985d54385bc4532fb19a265a5&role.id=bdbac30ca21b4146bf95b3a053dff30a => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:44.734228 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:44.753847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ca6d2-89af-47a2-b3f4-a9a2c6100eef None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:44.796732 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83427a-805e-4b9c-a49a-ada3e15bb26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:44.797430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83427a-805e-4b9c-a49a-ada3e15bb26f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:44.797746 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83427a-805e-4b9c-a49a-ada3e15bb26f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:44.798029 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83427a-805e-4b9c-a49a-ada3e15bb26f None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:44.798842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83427a-805e-4b9c-a49a-ada3e15bb26f None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:44.799156 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83427a-805e-4b9c-a49a-ada3e15bb26f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:44.800659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83427a-805e-4b9c-a49a-ada3e15bb26f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:44.814941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 123/250] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:14:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:44.820383 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75fda0-a5c6-4c3c-9a8f-b39b852160ae None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:44.853532 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:44.965309 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75fda0-a5c6-4c3c-9a8f-b39b852160ae None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:45.004752 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95243ee0-5f88-46b2-aeb2-791c1a9c3d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:45.005812 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95243ee0-5f88-46b2-aeb2-791c1a9c3d68 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:45.006122 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95243ee0-5f88-46b2-aeb2-791c1a9c3d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:45.006378 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95243ee0-5f88-46b2-aeb2-791c1a9c3d68 None admin] PATH_INFO: `/v3/users/96fabca16871409d86be75dcc35b51cc` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:45.008655 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95243ee0-5f88-46b2-aeb2-791c1a9c3d68 None admin] RBAC: Authorizing `identity:get_user(user_id=96fabca16871409d86be75dcc35b51cc)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:45.010150 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95243ee0-5f88-46b2-aeb2-791c1a9c3d68 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:45.012251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 128/251] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:44 2022] GET /identity/v3/users/96fabca16871409d86be75dcc35b51cc => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:45.018203 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83427a-805e-4b9c-a49a-ada3e15bb26f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:45.088878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6f907-3adf-4505-a981-57e87a5a9135 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:45.089146 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6f907-3adf-4505-a981-57e87a5a9135 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:45.089477 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6f907-3adf-4505-a981-57e87a5a9135 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:45.091405 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 129/252] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:45.097260 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2751b3e2-bfec-4853-9a83-01586817b3d6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:45.097650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2751b3e2-bfec-4853-9a83-01586817b3d6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:45.097906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2751b3e2-bfec-4853-9a83-01586817b3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:45.138864 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2751b3e2-bfec-4853-9a83-01586817b3d6 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:45.140126 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2751b3e2-bfec-4853-9a83-01586817b3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:45.278055 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 130/253] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 18:14:45.313859 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83427a-805e-4b9c-a49a-ada3e15bb26f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:45.358824 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2751b3e2-bfec-4853-9a83-01586817b3d6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:45.372918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32e0a3-7ab0-49ca-9750-e503bc6a3187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:45.374204 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32e0a3-7ab0-49ca-9750-e503bc6a3187 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:45.374714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32e0a3-7ab0-49ca-9750-e503bc6a3187 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:45.375190 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32e0a3-7ab0-49ca-9750-e503bc6a3187 None admin] PATH_INFO: `/v3/projects/9beede2985d54385bc4532fb19a265a5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:45.376844 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32e0a3-7ab0-49ca-9750-e503bc6a3187 None admin] RBAC: Authorizing `identity:get_project(project_id=9beede2985d54385bc4532fb19a265a5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:45.378708 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32e0a3-7ab0-49ca-9750-e503bc6a3187 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:45.380801 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 124/254] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:45 2022] GET /identity/v3/projects/9beede2985d54385bc4532fb19a265a5 => generated 302 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:45.386830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32e0a3-7ab0-49ca-9750-e503bc6a3187 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:45.405679 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:45.413280 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:45.551866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32e0a3-7ab0-49ca-9750-e503bc6a3187 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:45.597281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2751b3e2-bfec-4853-9a83-01586817b3d6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:45.627065 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933ae80-d289-4985-bbac-67bd4bf350b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:45.627893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933ae80-d289-4985-bbac-67bd4bf350b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:45.628173 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933ae80-d289-4985-bbac-67bd4bf350b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:45.628448 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933ae80-d289-4985-bbac-67bd4bf350b1 None admin] PATH_INFO: `/v3/projects/9beede2985d54385bc4532fb19a265a5/users/96fabca16871409d86be75dcc35b51cc/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:45.632174 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3933ae80-d289-4985-bbac-67bd4bf350b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9beede2985d54385bc4532fb19a265a5, user_id=96fabca16871409d86be75dcc35b51cc, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:45.634043 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3933ae80-d289-4985-bbac-67bd4bf350b1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:45.670814 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 125/255] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:14:45 2022] PUT /identity/v3/projects/9beede2985d54385bc4532fb19a265a5/users/96fabca16871409d86be75dcc35b51cc/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:14:45.682421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a07f9-2af7-4967-823f-f0d890a1078a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:45.683242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a07f9-2af7-4967-823f-f0d890a1078a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:45.683528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a07f9-2af7-4967-823f-f0d890a1078a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:45.683808 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a07f9-2af7-4967-823f-f0d890a1078a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:45.706536 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a07f9-2af7-4967-823f-f0d890a1078a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:45.715067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3c5a07f9-2af7-4967-823f-f0d890a1078a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:14:45.715067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:45.715067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:45.715067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:45.715067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:45.715067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:45.715067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:45.715067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:45.715067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:45.715067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:45.715067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:45.715067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:45.715067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:45.715067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:45.715067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:45.715067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:45.715967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:45.715967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:45.715967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:45.715967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:45.715967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:45.715967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:45.715967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:45.715967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:45.715967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:45.715967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:45.715967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:45.715967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:45.715967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:45.715967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:45.715967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:45.715967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:45.716741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:45.716741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:45.716741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:45.716741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:45.716741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:45.716741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:45.716741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:45.716741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:45.716741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:45.716741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:45.716741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:14:45.716741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:14:45.716741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 131/256] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:14:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:45.722657 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933ae80-d289-4985-bbac-67bd4bf350b1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:45.843099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:45.873743 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933ae80-d289-4985-bbac-67bd4bf350b1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:45.879092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a862dbd-fdcb-426c-ba7f-923b2a55d2ad None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:45.879362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a862dbd-fdcb-426c-ba7f-923b2a55d2ad None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:45.879575 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a862dbd-fdcb-426c-ba7f-923b2a55d2ad None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:45.880958 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 132/257] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:45.894023 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fc8ec-f0ed-4cc3-a0b4-46cadff1a143 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:45.894325 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fc8ec-f0ed-4cc3-a0b4-46cadff1a143 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:45.894585 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fc8ec-f0ed-4cc3-a0b4-46cadff1a143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:45.916285 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a15fc8ec-f0ed-4cc3-a0b4-46cadff1a143 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:45.917505 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a15fc8ec-f0ed-4cc3-a0b4-46cadff1a143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:45.938572 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e76d5-3d81-4349-887f-e312b20ee3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:45.939623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e76d5-3d81-4349-887f-e312b20ee3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:45.940103 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e76d5-3d81-4349-887f-e312b20ee3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:45.940573 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e76d5-3d81-4349-887f-e312b20ee3c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:45.941561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e76d5-3d81-4349-887f-e312b20ee3c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:45.942106 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e76d5-3d81-4349-887f-e312b20ee3c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:45.943741 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e76d5-3d81-4349-887f-e312b20ee3c6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:45.960959 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 126/258] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:45.966855 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e76d5-3d81-4349-887f-e312b20ee3c6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:46.041063 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 133/259] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 18:14:46.134894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e76d5-3d81-4349-887f-e312b20ee3c6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:46.139663 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fc8ec-f0ed-4cc3-a0b4-46cadff1a143 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:46.157997 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:46.227963 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e16481-7c71-4ea6-840f-324ebde6191f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:46.228852 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e16481-7c71-4ea6-840f-324ebde6191f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:46.229137 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e16481-7c71-4ea6-840f-324ebde6191f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:46.229418 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e16481-7c71-4ea6-840f-324ebde6191f None admin] PATH_INFO: `/v3/users/1d706385db2d4394aa1839ebfaaa37f5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:46.231025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e16481-7c71-4ea6-840f-324ebde6191f None admin] RBAC: Authorizing `identity:get_user(user_id=1d706385db2d4394aa1839ebfaaa37f5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:46.234379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e16481-7c71-4ea6-840f-324ebde6191f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:46.240769 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 127/260] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:45 2022] GET /identity/v3/users/1d706385db2d4394aa1839ebfaaa37f5 => generated 283 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:46.243207 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e16481-7c71-4ea6-840f-324ebde6191f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:46.267269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:46.379431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fc8ec-f0ed-4cc3-a0b4-46cadff1a143 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:46.402767 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e16481-7c71-4ea6-840f-324ebde6191f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:46.455874 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9d4c1-f124-433e-875a-0d78cd766d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:46.456541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9d4c1-f124-433e-875a-0d78cd766d27 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:46.456830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9d4c1-f124-433e-875a-0d78cd766d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:46.457112 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9d4c1-f124-433e-875a-0d78cd766d27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:46.464137 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53bd9e8-080d-4700-85d4-a495d34d405b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:46.464977 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bd9e8-080d-4700-85d4-a495d34d405b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:46.465266 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bd9e8-080d-4700-85d4-a495d34d405b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:46.465549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bd9e8-080d-4700-85d4-a495d34d405b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:46.490585 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53bd9e8-080d-4700-85d4-a495d34d405b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:46.511895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b53bd9e8-080d-4700-85d4-a495d34d405b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:14:46.511895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:46.511895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:46.511895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:46.511895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:46.511895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:46.511895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:46.511895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:46.511895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:46.511895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:46.511895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:46.511895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:46.511895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:46.511895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:46.511895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:46.511895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:46.514167 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:46.514167 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:46.514167 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:46.514167 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:46.514167 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:46.514167 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:46.514167 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:46.514167 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:46.514167 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:46.514167 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:46.514167 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:46.514167 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:46.514167 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:46.514167 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:46.514167 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:46.514167 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:46.516357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:46.516357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:46.516357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:46.516357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:46.516357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:46.516357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:46.516357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:46.516357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:46.516357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:46.516357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:46.516357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:14:46.516357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:14:46.517872 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 134/261] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 30 18:14:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:46.544287 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53bd9e8-080d-4700-85d4-a495d34d405b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:46.618332 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9d4c1-f124-433e-875a-0d78cd766d27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:46.619852 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9d4c1-f124-433e-875a-0d78cd766d27 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:46.654234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:46.681990 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:46.778713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 128/262] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:14:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:14:46.788844 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9d4c1-f124-433e-875a-0d78cd766d27 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:46.795921 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53bd9e8-080d-4700-85d4-a495d34d405b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:46.886135 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e3fc4-b4c2-439d-a4bb-582cd459c7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:46.886808 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e3fc4-b4c2-439d-a4bb-582cd459c7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:46.887093 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e3fc4-b4c2-439d-a4bb-582cd459c7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:46.887478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e3fc4-b4c2-439d-a4bb-582cd459c7b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:46.888238 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90e3fc4-b4c2-439d-a4bb-582cd459c7b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:46.888545 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90e3fc4-b4c2-439d-a4bb-582cd459c7b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:46.889999 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90e3fc4-b4c2-439d-a4bb-582cd459c7b0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:46.910769 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 135/263] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:14:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:46.918013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e3fc4-b4c2-439d-a4bb-582cd459c7b0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:46.965875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9d4c1-f124-433e-875a-0d78cd766d27 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:46.975005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:47.032163 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e3fc4-b4c2-439d-a4bb-582cd459c7b0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:47.034045 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157153d-6773-4b14-aa60-3091237edde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:47.034918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157153d-6773-4b14-aa60-3091237edde5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:47.035200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157153d-6773-4b14-aa60-3091237edde5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:47.035481 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157153d-6773-4b14-aa60-3091237edde5 None admin] PATH_INFO: `/v3/projects/9beede2985d54385bc4532fb19a265a5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:47.042446 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157153d-6773-4b14-aa60-3091237edde5 None admin] RBAC: Authorizing `identity:get_project(project_id=9beede2985d54385bc4532fb19a265a5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:47.044183 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157153d-6773-4b14-aa60-3091237edde5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:47.046033 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 129/264] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:46 2022] GET /identity/v3/projects/9beede2985d54385bc4532fb19a265a5 => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:47.052226 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157153d-6773-4b14-aa60-3091237edde5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:47.062924 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:47.082815 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67943af7-c55e-4c9f-bcbe-5f39839510bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:47.084827 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67943af7-c55e-4c9f-bcbe-5f39839510bb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:47.086271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67943af7-c55e-4c9f-bcbe-5f39839510bb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:47.087024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67943af7-c55e-4c9f-bcbe-5f39839510bb None admin] PATH_INFO: `/v3/users/1d706385db2d4394aa1839ebfaaa37f5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:47.090977 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67943af7-c55e-4c9f-bcbe-5f39839510bb None admin] RBAC: Authorizing `identity:get_user(user_id=1d706385db2d4394aa1839ebfaaa37f5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:47.094171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67943af7-c55e-4c9f-bcbe-5f39839510bb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:47.106227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 136/265] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:46 2022] GET /identity/v3/users/1d706385db2d4394aa1839ebfaaa37f5 => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:47.112140 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67943af7-c55e-4c9f-bcbe-5f39839510bb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:47.237274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157153d-6773-4b14-aa60-3091237edde5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:47.312622 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59dda1-a31b-49ea-8635-319b366460c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:47.313305 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59dda1-a31b-49ea-8635-319b366460c0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:47.313619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59dda1-a31b-49ea-8635-319b366460c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:47.313907 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59dda1-a31b-49ea-8635-319b366460c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:47.314388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67943af7-c55e-4c9f-bcbe-5f39839510bb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:47.314758 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59dda1-a31b-49ea-8635-319b366460c0 None admin] RBAC: Adding query filter params (role.id=0ef3687af20a443281e3cfdceca3597d, scope.project.id=9beede2985d54385bc4532fb19a265a5, user.id=1d706385db2d4394aa1839ebfaaa37f5) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:47.315080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59dda1-a31b-49ea-8635-319b366460c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:47.316785 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59dda1-a31b-49ea-8635-319b366460c0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:47.337527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 130/266] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:14:47 2022] GET /identity/v3/role_assignments?user.id=1d706385db2d4394aa1839ebfaaa37f5&scope.project.id=9beede2985d54385bc4532fb19a265a5&role.id=0ef3687af20a443281e3cfdceca3597d => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:47.360999 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2e510-b680-47e2-afe9-082d4fbf5f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:47.361740 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2e510-b680-47e2-afe9-082d4fbf5f3b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:47.362028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2e510-b680-47e2-afe9-082d4fbf5f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:47.362305 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2e510-b680-47e2-afe9-082d4fbf5f3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:47.432264 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:47.442667 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f2e510-b680-47e2-afe9-082d4fbf5f3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:47.444170 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f2e510-b680-47e2-afe9-082d4fbf5f3b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:47.598161 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 137/267] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:14:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 18:14:47.604010 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59dda1-a31b-49ea-8635-319b366460c0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:47.658362 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:47.669792 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f75e2-7672-4d4d-b914-012958298add None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:47.670106 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f75e2-7672-4d4d-b914-012958298add None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:47.670367 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f75e2-7672-4d4d-b914-012958298add None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:47.671794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 138/268] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:47.677498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eab3d7-28c8-4cd9-b9a4-9a188f164e4b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:47.677830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eab3d7-28c8-4cd9-b9a4-9a188f164e4b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:47.678166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eab3d7-28c8-4cd9-b9a4-9a188f164e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:47.696769 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-19eab3d7-28c8-4cd9-b9a4-9a188f164e4b None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:47.698022 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-19eab3d7-28c8-4cd9-b9a4-9a188f164e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:47.780250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 139/269] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 18:14:47.796418 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59dda1-a31b-49ea-8635-319b366460c0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:47.808013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eab3d7-28c8-4cd9-b9a4-9a188f164e4b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:47.854751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:47.858176 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9e093-8ddf-40b2-8169-4c87559e5d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:47.858176 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9e093-8ddf-40b2-8169-4c87559e5d10 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:47.858176 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9e093-8ddf-40b2-8169-4c87559e5d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:47.858176 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9e093-8ddf-40b2-8169-4c87559e5d10 None admin] PATH_INFO: `/v3/projects/9beede2985d54385bc4532fb19a265a5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:47.859860 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9e093-8ddf-40b2-8169-4c87559e5d10 None admin] RBAC: Authorizing `identity:get_project(project_id=9beede2985d54385bc4532fb19a265a5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:47.868400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9e093-8ddf-40b2-8169-4c87559e5d10 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:47.871825 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 131/270] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:47 2022] GET /identity/v3/projects/9beede2985d54385bc4532fb19a265a5 => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:47.877409 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9e093-8ddf-40b2-8169-4c87559e5d10 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:47.963604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eab3d7-28c8-4cd9-b9a4-9a188f164e4b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:48.008108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076647ef-618a-40c5-a7ed-db1d247980d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:48.009472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076647ef-618a-40c5-a7ed-db1d247980d9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:48.010031 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076647ef-618a-40c5-a7ed-db1d247980d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:48.010512 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076647ef-618a-40c5-a7ed-db1d247980d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:48.023603 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076647ef-618a-40c5-a7ed-db1d247980d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:48.033227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-076647ef-618a-40c5-a7ed-db1d247980d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:14:48.033227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:48.033227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:48.033227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:48.033227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:48.033227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:48.033227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:48.033227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:48.033227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:48.033227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:48.033227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:48.033227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:48.033227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:48.033227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:48.033227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:48.033227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:48.034429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:48.034429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:48.034429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:48.034429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:48.034429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:48.034429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:48.034429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:48.034429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:48.034429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:48.034429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:48.034429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:48.034429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:48.034429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:48.034429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:48.034429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:48.034429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:48.035222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:48.035222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:48.035222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:48.035222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:48.035222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:48.035222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:48.035222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:48.035222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:48.035222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:48.035222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:48.035222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:14:48.035222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:14:48.037646 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 140/271] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:14:48.040149 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53ca4d-d69e-4665-8058-fa20fed77b84 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:48.040690 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53ca4d-d69e-4665-8058-fa20fed77b84 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:48.041190 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53ca4d-d69e-4665-8058-fa20fed77b84 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:48.042819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 141/272] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:48.055391 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53ca4d-d69e-4665-8058-fa20fed77b84 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:48.063728 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9e093-8ddf-40b2-8169-4c87559e5d10 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:48.075087 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:48.113160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f840d8-3f0b-46b7-b010-eec3ab96320b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:48.113920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f840d8-3f0b-46b7-b010-eec3ab96320b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:48.114755 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f840d8-3f0b-46b7-b010-eec3ab96320b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:48.115118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f840d8-3f0b-46b7-b010-eec3ab96320b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:48.116071 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f840d8-3f0b-46b7-b010-eec3ab96320b None admin] RBAC: Adding query filter params (role.id=d5f874a3d78f4ee0ade318e1bbebf7ec, scope.project.id=9beede2985d54385bc4532fb19a265a5, user.id=1d706385db2d4394aa1839ebfaaa37f5) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:48.117097 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f840d8-3f0b-46b7-b010-eec3ab96320b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:48.119402 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f840d8-3f0b-46b7-b010-eec3ab96320b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:48.128231 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 132/273] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:14:47 2022] GET /identity/v3/role_assignments?user.id=1d706385db2d4394aa1839ebfaaa37f5&scope.project.id=9beede2985d54385bc4532fb19a265a5&role.id=d5f874a3d78f4ee0ade318e1bbebf7ec => generated 608 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:48.129701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d7c84-95a9-4324-a8cc-88c02961803a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:48.130117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d7c84-95a9-4324-a8cc-88c02961803a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:48.130478 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d7c84-95a9-4324-a8cc-88c02961803a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:48.177921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-332d7c84-95a9-4324-a8cc-88c02961803a None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:48.180469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-332d7c84-95a9-4324-a8cc-88c02961803a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:48.265252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:48.293964 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53ca4d-d69e-4665-8058-fa20fed77b84 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:48.313415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 133/274] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:48.316366 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d6376-e50f-4197-b961-20630ce84579 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:48.316660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d6376-e50f-4197-b961-20630ce84579 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:48.316943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d6376-e50f-4197-b961-20630ce84579 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:48.318771 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 134/275] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:48.331123 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1c739-6b20-4abe-8870-1d65caecd3cb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:48.331411 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1c739-6b20-4abe-8870-1d65caecd3cb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:48.331719 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1c739-6b20-4abe-8870-1d65caecd3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:48.356766 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-47c1c739-6b20-4abe-8870-1d65caecd3cb None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:48.358036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-47c1c739-6b20-4abe-8870-1d65caecd3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:48.359646 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d3068-4dcd-4407-af8c-4e2469c8126d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:48.361506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d3068-4dcd-4407-af8c-4e2469c8126d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:48.362132 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d3068-4dcd-4407-af8c-4e2469c8126d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:48.362682 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d3068-4dcd-4407-af8c-4e2469c8126d None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:48.364230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d3068-4dcd-4407-af8c-4e2469c8126d None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:48.364781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d3068-4dcd-4407-af8c-4e2469c8126d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:48.368050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d3068-4dcd-4407-af8c-4e2469c8126d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:48.381272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 142/276] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:14:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:48.387248 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d3068-4dcd-4407-af8c-4e2469c8126d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:48.452851 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 135/277] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 18:14:48.462738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1c739-6b20-4abe-8870-1d65caecd3cb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:48.532572 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:48.566441 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d3068-4dcd-4407-af8c-4e2469c8126d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:48.606870 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fa193-5be6-4940-8579-36d61975aa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:48.607680 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fa193-5be6-4940-8579-36d61975aa8b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:48.607971 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fa193-5be6-4940-8579-36d61975aa8b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:48.608255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fa193-5be6-4940-8579-36d61975aa8b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:48.620901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08fa193-5be6-4940-8579-36d61975aa8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:48.628726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a08fa193-5be6-4940-8579-36d61975aa8b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 18:14:48.628726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:48.628726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:48.628726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:48.628726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:48.628726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:48.628726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:48.628726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:48.628726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:48.628726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:48.628726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:48.628726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:48.628726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:48.628726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:48.628726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:48.628726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:48.629748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:48.629748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:48.629748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:48.629748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:48.629748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:48.629748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:48.629748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:48.629748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:48.629748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:48.629748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:48.629748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:48.629748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:48.629748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:48.629748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:48.629748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:48.629748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:48.630582 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:48.630582 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:48.630582 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:48.630582 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:48.630582 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:48.630582 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:48.630582 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:48.630582 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:48.630582 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:48.630582 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:48.630582 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 18:14:48.630582 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:14:48.630582 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 143/278] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:48.631295 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fa193-5be6-4940-8579-36d61975aa8b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:48.645794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:48.646865 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1c739-6b20-4abe-8870-1d65caecd3cb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:48.706594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abfbdc-6193-475a-8876-3df26515f59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:48.707489 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abfbdc-6193-475a-8876-3df26515f59f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:48.707774 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abfbdc-6193-475a-8876-3df26515f59f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:48.708052 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abfbdc-6193-475a-8876-3df26515f59f None admin] PATH_INFO: `/v3/users/24310fb42a0947cd81738be84d5f53ef` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:48.724421 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abfbdc-6193-475a-8876-3df26515f59f None admin] RBAC: Authorizing `identity:get_user(user_id=24310fb42a0947cd81738be84d5f53ef)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:48.726844 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abfbdc-6193-475a-8876-3df26515f59f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:48.733706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 136/279] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:48 2022] GET /identity/v3/users/24310fb42a0947cd81738be84d5f53ef => generated 293 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:48.733706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abfbdc-6193-475a-8876-3df26515f59f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:48.776916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fa193-5be6-4940-8579-36d61975aa8b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:48.828437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6deaa2-1f63-4434-83b4-5a833d11bc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:48.829567 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6deaa2-1f63-4434-83b4-5a833d11bc30 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:48.829567 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6deaa2-1f63-4434-83b4-5a833d11bc30 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:48.830013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6deaa2-1f63-4434-83b4-5a833d11bc30 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:48.831122 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6deaa2-1f63-4434-83b4-5a833d11bc30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:48.832648 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6deaa2-1f63-4434-83b4-5a833d11bc30 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:48.834522 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 144/280] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:14:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:48.835305 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6deaa2-1f63-4434-83b4-5a833d11bc30 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:48.865292 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:48.882830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abfbdc-6193-475a-8876-3df26515f59f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:48.923106 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19278f01-e763-40fe-88d5-f877243b1c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:48.923904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19278f01-e763-40fe-88d5-f877243b1c8b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:48.924198 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19278f01-e763-40fe-88d5-f877243b1c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:48.924479 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19278f01-e763-40fe-88d5-f877243b1c8b None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:48.925371 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19278f01-e763-40fe-88d5-f877243b1c8b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:48.925720 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19278f01-e763-40fe-88d5-f877243b1c8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:48.927140 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19278f01-e763-40fe-88d5-f877243b1c8b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:48.931847 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:48.943889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 137/281] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:14:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:48.943889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19278f01-e763-40fe-88d5-f877243b1c8b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:48.967095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6deaa2-1f63-4434-83b4-5a833d11bc30 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:49.007008 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c73383-5aa8-475d-a2c5-8255447e0018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:49.008086 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c73383-5aa8-475d-a2c5-8255447e0018 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:49.008400 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c73383-5aa8-475d-a2c5-8255447e0018 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:49.008825 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c73383-5aa8-475d-a2c5-8255447e0018 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:49.010203 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c73383-5aa8-475d-a2c5-8255447e0018 None admin] RBAC: Adding query filter params (role.id=3fbb71ae2de04e78b68d6d6f407ef09c, user.id=24310fb42a0947cd81738be84d5f53ef, scope.system=all) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:49.010698 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c73383-5aa8-475d-a2c5-8255447e0018 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:49.012987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c73383-5aa8-475d-a2c5-8255447e0018 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:49.029612 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 145/282] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Jun 30 18:14:48 2022] GET /identity/v3/role_assignments?user.id=24310fb42a0947cd81738be84d5f53ef&scope.system=all&role.id=3fbb71ae2de04e78b68d6d6f407ef09c => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:49.029612 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c73383-5aa8-475d-a2c5-8255447e0018 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:49.092201 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19278f01-e763-40fe-88d5-f877243b1c8b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:49.148748 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd644601-dc46-442a-8476-e9731e25e508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:49.149674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd644601-dc46-442a-8476-e9731e25e508 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:49.149674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd644601-dc46-442a-8476-e9731e25e508 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:49.149952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd644601-dc46-442a-8476-e9731e25e508 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:49.150710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd644601-dc46-442a-8476-e9731e25e508 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:49.152252 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd644601-dc46-442a-8476-e9731e25e508 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:49.181238 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 138/283] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 30 18:14:48 2022] POST /identity/v3/users => generated 293 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:14:49.183979 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65924c2d-d95e-424e-b324-92a245561e8d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:49.184464 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65924c2d-d95e-424e-b324-92a245561e8d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:49.184998 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65924c2d-d95e-424e-b324-92a245561e8d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:49.187649 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 139/284] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:49.192924 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c73383-5aa8-475d-a2c5-8255447e0018 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:49.196077 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c7deb-3000-4c4f-8ddd-a0d113c76bef None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:49.196530 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c7deb-3000-4c4f-8ddd-a0d113c76bef None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:49.196971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c7deb-3000-4c4f-8ddd-a0d113c76bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:49.217319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:49.231414 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b93c7deb-3000-4c4f-8ddd-a0d113c76bef None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:49.233030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b93c7deb-3000-4c4f-8ddd-a0d113c76bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:49.268501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e86887-efd5-4d00-bc74-181b04377349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:49.269400 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e86887-efd5-4d00-bc74-181b04377349 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:49.269716 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e86887-efd5-4d00-bc74-181b04377349 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:49.269989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e86887-efd5-4d00-bc74-181b04377349 None admin] PATH_INFO: `/v3/users/e98c10b29c0d4866b532931fd1855e0c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:49.272314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e86887-efd5-4d00-bc74-181b04377349 None admin] RBAC: Authorizing `identity:get_user(user_id=e98c10b29c0d4866b532931fd1855e0c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:49.274531 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e86887-efd5-4d00-bc74-181b04377349 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:49.276489 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 146/285] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:49 2022] GET /identity/v3/users/e98c10b29c0d4866b532931fd1855e0c => generated 297 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:49.282620 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e86887-efd5-4d00-bc74-181b04377349 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:49.295829 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:49.391493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 140/286] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:49.426025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c7deb-3000-4c4f-8ddd-a0d113c76bef None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:49.521470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e86887-efd5-4d00-bc74-181b04377349 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:49.576129 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ec55c-497f-4216-ac4f-bd53ceab0edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:49.577688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ec55c-497f-4216-ac4f-bd53ceab0edc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:49.578139 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ec55c-497f-4216-ac4f-bd53ceab0edc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:49.578651 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ec55c-497f-4216-ac4f-bd53ceab0edc None admin] PATH_INFO: `/v3/projects/9beede2985d54385bc4532fb19a265a5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:49.580094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ec55c-497f-4216-ac4f-bd53ceab0edc None admin] RBAC: Authorizing `identity:get_project(project_id=9beede2985d54385bc4532fb19a265a5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:49.581873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ec55c-497f-4216-ac4f-bd53ceab0edc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:49.583793 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 147/287] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:49 2022] GET /identity/v3/projects/9beede2985d54385bc4532fb19a265a5 => generated 302 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:49.589906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ec55c-497f-4216-ac4f-bd53ceab0edc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:49.624869 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c7deb-3000-4c4f-8ddd-a0d113c76bef None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:49.676423 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1184d-202c-4b91-8d1a-aed2d044ba71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:49.677161 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1184d-202c-4b91-8d1a-aed2d044ba71 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:49.677446 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1184d-202c-4b91-8d1a-aed2d044ba71 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:49.677751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1184d-202c-4b91-8d1a-aed2d044ba71 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:49.682841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:49.696245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1184d-202c-4b91-8d1a-aed2d044ba71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:49.704108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:49.706347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4ca1184d-202c-4b91-8d1a-aed2d044ba71 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:14:49.706347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:49.706347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:49.706347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:49.706347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:49.706347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:49.706347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:49.706347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:49.706347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:49.706347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:49.706347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:49.706347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:49.706347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:49.706347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:49.706347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:49.706347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:49.707254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:49.707254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:49.707254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:49.707254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:49.707254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:49.707254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:49.707254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:49.707254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:49.707254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:49.707254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:49.707254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:49.707254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:49.707254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:49.707254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:49.707254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:49.707254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:49.708040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:49.708040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:49.708040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:49.708040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:49.708040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:49.708040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:49.708040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:49.708040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:49.708040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:49.708040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:49.708040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:14:49.708040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:14:49.708625 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 141/288] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:14:49.716110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1184d-202c-4b91-8d1a-aed2d044ba71 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:49.747130 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ec55c-497f-4216-ac4f-bd53ceab0edc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:49.789465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492d3df-713c-4efb-9398-f81d4caa8726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:49.790304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492d3df-713c-4efb-9398-f81d4caa8726 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:49.790709 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492d3df-713c-4efb-9398-f81d4caa8726 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:49.790995 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492d3df-713c-4efb-9398-f81d4caa8726 None admin] PATH_INFO: `/v3/projects/9beede2985d54385bc4532fb19a265a5/users/e98c10b29c0d4866b532931fd1855e0c/roles/bdbac30ca21b4146bf95b3a053dff30a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:49.793117 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492d3df-713c-4efb-9398-f81d4caa8726 None admin] RBAC: Authorizing `identity:create_grant(project_id=9beede2985d54385bc4532fb19a265a5, user_id=e98c10b29c0d4866b532931fd1855e0c, role_id=bdbac30ca21b4146bf95b3a053dff30a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:49.794908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492d3df-713c-4efb-9398-f81d4caa8726 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:49.820649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 148/289] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:14:49 2022] PUT /identity/v3/projects/9beede2985d54385bc4532fb19a265a5/users/e98c10b29c0d4866b532931fd1855e0c/roles/bdbac30ca21b4146bf95b3a053dff30a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:14:49.868667 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1184d-202c-4b91-8d1a-aed2d044ba71 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:49.914862 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0206a9-8d67-49fa-bc93-0261372bcd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:49.915987 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0206a9-8d67-49fa-bc93-0261372bcd2f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:49.916553 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0206a9-8d67-49fa-bc93-0261372bcd2f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:49.917018 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0206a9-8d67-49fa-bc93-0261372bcd2f None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:49.918511 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0206a9-8d67-49fa-bc93-0261372bcd2f None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:49.919313 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0206a9-8d67-49fa-bc93-0261372bcd2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:49.922089 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0206a9-8d67-49fa-bc93-0261372bcd2f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:49.936731 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 142/290] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:14:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:49.943897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492d3df-713c-4efb-9398-f81d4caa8726 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:50.092121 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492d3df-713c-4efb-9398-f81d4caa8726 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:50.144725 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c4586-be91-4296-b4ce-9486c4581afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:50.146813 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c4586-be91-4296-b4ce-9486c4581afc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:50.146813 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c4586-be91-4296-b4ce-9486c4581afc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:50.146813 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c4586-be91-4296-b4ce-9486c4581afc None admin] PATH_INFO: `/v3/users/96fabca16871409d86be75dcc35b51cc` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:50.148821 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c4586-be91-4296-b4ce-9486c4581afc None admin] RBAC: Authorizing `identity:get_user(user_id=96fabca16871409d86be75dcc35b51cc)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:50.150382 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c4586-be91-4296-b4ce-9486c4581afc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:50.152187 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 149/291] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:49 2022] GET /identity/v3/users/96fabca16871409d86be75dcc35b51cc => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:50.158571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0206a9-8d67-49fa-bc93-0261372bcd2f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:50.236548 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:50.287915 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0206a9-8d67-49fa-bc93-0261372bcd2f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:50.330498 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588140c-5a8c-4f3b-8ffe-6d33606eadb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:50.331162 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588140c-5a8c-4f3b-8ffe-6d33606eadb2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:50.331498 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588140c-5a8c-4f3b-8ffe-6d33606eadb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:50.331758 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588140c-5a8c-4f3b-8ffe-6d33606eadb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:50.410029 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7588140c-5a8c-4f3b-8ffe-6d33606eadb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:50.411528 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7588140c-5a8c-4f3b-8ffe-6d33606eadb2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:50.481169 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:50.536900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 143/292] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:14:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:14:50.543458 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c4586-be91-4296-b4ce-9486c4581afc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:50.546797 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:50.708912 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c4586-be91-4296-b4ce-9486c4581afc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:50.757935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4441f31-cb77-4ab6-ba30-79038255bde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:50.759132 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4441f31-cb77-4ab6-ba30-79038255bde6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:50.759560 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4441f31-cb77-4ab6-ba30-79038255bde6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:50.759981 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4441f31-cb77-4ab6-ba30-79038255bde6 None admin] PATH_INFO: `/v3/projects/9beede2985d54385bc4532fb19a265a5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:50.761425 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4441f31-cb77-4ab6-ba30-79038255bde6 None admin] RBAC: Authorizing `identity:get_project(project_id=9beede2985d54385bc4532fb19a265a5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:50.763086 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4441f31-cb77-4ab6-ba30-79038255bde6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:50.765027 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 150/293] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:50 2022] GET /identity/v3/projects/9beede2985d54385bc4532fb19a265a5 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:50.772261 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588140c-5a8c-4f3b-8ffe-6d33606eadb2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:50.973015 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588140c-5a8c-4f3b-8ffe-6d33606eadb2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:51.031374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f4fca-a0d1-4fcb-bfee-a9f81281edb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:51.032053 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f4fca-a0d1-4fcb-bfee-a9f81281edb7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:51.032339 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f4fca-a0d1-4fcb-bfee-a9f81281edb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:51.032618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f4fca-a0d1-4fcb-bfee-a9f81281edb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:51.033474 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f4fca-a0d1-4fcb-bfee-a9f81281edb7 None admin] RBAC: Adding query filter params (role.id=3fbb71ae2de04e78b68d6d6f407ef09c, scope.project.id=9beede2985d54385bc4532fb19a265a5, user.id=96fabca16871409d86be75dcc35b51cc) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:51.033825 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f4fca-a0d1-4fcb-bfee-a9f81281edb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:51.035304 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f4fca-a0d1-4fcb-bfee-a9f81281edb7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:51.046963 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 144/294] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:14:50 2022] GET /identity/v3/role_assignments?user.id=96fabca16871409d86be75dcc35b51cc&scope.project.id=9beede2985d54385bc4532fb19a265a5&role.id=3fbb71ae2de04e78b68d6d6f407ef09c => generated 608 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:51.544811 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68edeb-8bda-4df5-a0cc-cdc72e742a8b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:51.545244 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68edeb-8bda-4df5-a0cc-cdc72e742a8b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:51.545506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68edeb-8bda-4df5-a0cc-cdc72e742a8b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:51.548588 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 151/295] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:51.555520 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a2a0f-797a-4143-bed1-dfc48d11b6f0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:51.556471 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a2a0f-797a-4143-bed1-dfc48d11b6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:51.556975 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a2a0f-797a-4143-bed1-dfc48d11b6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:51.575147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ee3a2a0f-797a-4143-bed1-dfc48d11b6f0 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:51.576683 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ee3a2a0f-797a-4143-bed1-dfc48d11b6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:51.596030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:51.653673 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 145/296] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:51.678275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68edeb-8bda-4df5-a0cc-cdc72e742a8b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:51.839428 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:51.896170 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68edeb-8bda-4df5-a0cc-cdc72e742a8b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:51.946947 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b597ee1-ee3b-48bd-b454-540ba57eacf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:51.947820 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b597ee1-ee3b-48bd-b454-540ba57eacf3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:51.948108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b597ee1-ee3b-48bd-b454-540ba57eacf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:51.948387 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b597ee1-ee3b-48bd-b454-540ba57eacf3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:51.960612 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b597ee1-ee3b-48bd-b454-540ba57eacf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:51.969110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4b597ee1-ee3b-48bd-b454-540ba57eacf3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:14:51.969110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:51.969110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:51.969110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:51.969110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:51.969110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:51.969110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:51.969110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:51.969110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:51.969110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:51.969110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:51.969110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:51.969110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:51.969110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:51.969110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:51.969110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:51.970225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:51.970225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:51.970225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:51.970225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:51.970225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:51.970225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:51.970225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:51.970225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:51.970225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:51.970225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:51.970225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:51.970225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:51.970225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:51.970225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:51.970225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:51.970225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:51.971199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:51.971199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:51.971199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:51.971199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:51.971199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:51.971199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:51.971199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:51.971199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:51.971199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:51.971199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:51.971199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:14:51.971199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:14:51.973254 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 152/297] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:51.980037 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a2a0f-797a-4143-bed1-dfc48d11b6f0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:52.097279 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a2a0f-797a-4143-bed1-dfc48d11b6f0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:52.136042 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076a082-f74a-4dde-9bb8-bd76f1d47dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:52.137078 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076a082-f74a-4dde-9bb8-bd76f1d47dff None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:52.137629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076a082-f74a-4dde-9bb8-bd76f1d47dff None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:52.138109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076a082-f74a-4dde-9bb8-bd76f1d47dff None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:52.139074 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076a082-f74a-4dde-9bb8-bd76f1d47dff None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:52.139580 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076a082-f74a-4dde-9bb8-bd76f1d47dff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:52.141310 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076a082-f74a-4dde-9bb8-bd76f1d47dff None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:52.152107 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 146/298] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:14:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:52.162781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b597ee1-ee3b-48bd-b454-540ba57eacf3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:52.309195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b597ee1-ee3b-48bd-b454-540ba57eacf3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:52.340396 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85017f1-edae-44a0-831a-96a3ebd0fe05 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:52.340709 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85017f1-edae-44a0-831a-96a3ebd0fe05 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:52.340964 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85017f1-edae-44a0-831a-96a3ebd0fe05 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:52.342374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 147/299] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:52.345673 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:52.354020 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb2bf9-11d3-436e-b8df-85245b8ddfa6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:52.354624 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb2bf9-11d3-436e-b8df-85245b8ddfa6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:52.355138 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb2bf9-11d3-436e-b8df-85245b8ddfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:52.357706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e83a3-9b90-448b-96f7-0ec7468c2905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:52.359256 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e83a3-9b90-448b-96f7-0ec7468c2905 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:52.359754 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e83a3-9b90-448b-96f7-0ec7468c2905 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:52.360257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e83a3-9b90-448b-96f7-0ec7468c2905 None admin] PATH_INFO: `/v3/users/24310fb42a0947cd81738be84d5f53ef` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:52.363902 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e83a3-9b90-448b-96f7-0ec7468c2905 None admin] RBAC: Authorizing `identity:get_user(user_id=24310fb42a0947cd81738be84d5f53ef)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:52.365481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e83a3-9b90-448b-96f7-0ec7468c2905 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:52.369633 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 153/300] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:52 2022] GET /identity/v3/users/24310fb42a0947cd81738be84d5f53ef => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:52.374602 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e83a3-9b90-448b-96f7-0ec7468c2905 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:52.377412 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-39fb2bf9-11d3-436e-b8df-85245b8ddfa6 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:52.380457 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-39fb2bf9-11d3-436e-b8df-85245b8ddfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:52.404246 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:52.473183 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 148/301] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 18:14:52.523055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb2bf9-11d3-436e-b8df-85245b8ddfa6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:52.557377 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e83a3-9b90-448b-96f7-0ec7468c2905 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:52.598938 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5865b-3624-42c5-8f74-b2062dfb15f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:52.600248 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5865b-3624-42c5-8f74-b2062dfb15f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:52.601174 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5865b-3624-42c5-8f74-b2062dfb15f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:52.602368 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5865b-3624-42c5-8f74-b2062dfb15f0 None admin] PATH_INFO: `/v3/system/users/24310fb42a0947cd81738be84d5f53ef/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:52.603732 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a5865b-3624-42c5-8f74-b2062dfb15f0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=24310fb42a0947cd81738be84d5f53ef, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:52.605639 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a5865b-3624-42c5-8f74-b2062dfb15f0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:52.616004 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 154/302] 10.222.0.33 () {58 vars in 1334 bytes} [Thu Jun 30 18:14:52 2022] PUT /identity/v3/system/users/24310fb42a0947cd81738be84d5f53ef/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:14:52.618223 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339046df-720b-462c-a46e-3127f38b0ed8 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:52.618515 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339046df-720b-462c-a46e-3127f38b0ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:52.618792 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339046df-720b-462c-a46e-3127f38b0ed8 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:52.620678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 155/303] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:52.621821 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0fea9-cc07-4062-a40d-215f6fcbd459 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:52.622142 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0fea9-cc07-4062-a40d-215f6fcbd459 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:52.622452 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0fea9-cc07-4062-a40d-215f6fcbd459 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:52.624348 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 156/304] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:52.630366 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838abf6f-9688-4f67-89cd-1a0c3ff585de None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:52.630737 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838abf6f-9688-4f67-89cd-1a0c3ff585de None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:52.630962 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838abf6f-9688-4f67-89cd-1a0c3ff585de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:52.659666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-838abf6f-9688-4f67-89cd-1a0c3ff585de None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:52.661721 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-838abf6f-9688-4f67-89cd-1a0c3ff585de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:52.710496 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb2bf9-11d3-436e-b8df-85245b8ddfa6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:52.759530 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808b4bf-8bd2-4a16-a210-9b921c0b5ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:52.760354 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808b4bf-8bd2-4a16-a210-9b921c0b5ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:52.760645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808b4bf-8bd2-4a16-a210-9b921c0b5ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:52.760921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808b4bf-8bd2-4a16-a210-9b921c0b5ef1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:52.774735 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3808b4bf-8bd2-4a16-a210-9b921c0b5ef1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:52.776514 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:52.783821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3808b4bf-8bd2-4a16-a210-9b921c0b5ef1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 18:14:52.783821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:52.783821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:52.783821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:52.783821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:52.783821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:52.783821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:52.783821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:52.783821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:52.783821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:52.783821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:52.783821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:52.783821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:52.783821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:52.783821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:52.783821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:52.785022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:52.785022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:52.785022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:52.785022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:52.785022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:52.785022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:52.785022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:52.785022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:52.785022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:52.785022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:52.785022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:52.785022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:52.785022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:52.785022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:52.785022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:52.785022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:52.786245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:52.786245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:52.786245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:52.786245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:52.786245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:52.786245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:52.786245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:52.786245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:52.786245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:52.786245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:52.786245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 18:14:52.786245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:14:52.786245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 149/305] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:14:52.786245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b26909-2c6d-498b-825e-3396996b52e6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:52.786981 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b26909-2c6d-498b-825e-3396996b52e6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:52.787142 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b26909-2c6d-498b-825e-3396996b52e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:52.800227 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:52.806126 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 157/306] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:52.807101 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838abf6f-9688-4f67-89cd-1a0c3ff585de None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:52.814926 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-67b26909-2c6d-498b-825e-3396996b52e6 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:52.816798 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-67b26909-2c6d-498b-825e-3396996b52e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:52.904579 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 150/307] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:52.906028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b26909-2c6d-498b-825e-3396996b52e6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:52.937804 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838abf6f-9688-4f67-89cd-1a0c3ff585de None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:52.977175 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102f9e1-b395-4849-a034-f0bebbba3f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:52.977892 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102f9e1-b395-4849-a034-f0bebbba3f21 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:52.978180 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102f9e1-b395-4849-a034-f0bebbba3f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:52.978460 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102f9e1-b395-4849-a034-f0bebbba3f21 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:52.979226 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8102f9e1-b395-4849-a034-f0bebbba3f21 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:52.979537 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8102f9e1-b395-4849-a034-f0bebbba3f21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:52.980977 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8102f9e1-b395-4849-a034-f0bebbba3f21 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:52.990433 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 158/308] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:14:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:52.992007 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102f9e1-b395-4849-a034-f0bebbba3f21 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:53.074677 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b26909-2c6d-498b-825e-3396996b52e6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:53.086029 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:53.136224 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8240b5e4-96eb-41f7-b750-e1bea1c35c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:53.137284 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240b5e4-96eb-41f7-b750-e1bea1c35c6b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:53.137606 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240b5e4-96eb-41f7-b750-e1bea1c35c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:53.137883 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240b5e4-96eb-41f7-b750-e1bea1c35c6b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:53.140163 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:53.151387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8240b5e4-96eb-41f7-b750-e1bea1c35c6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:53.159745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-8240b5e4-96eb-41f7-b750-e1bea1c35c6b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 18:14:53.159745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:53.159745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:53.159745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:53.159745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:53.159745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:53.159745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:53.159745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:53.159745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:53.159745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:53.159745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:53.159745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:53.159745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:53.159745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:53.159745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:53.159745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:53.160770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:53.160770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:53.160770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:53.160770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:53.160770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:53.160770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:53.160770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:53.160770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:53.160770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:53.160770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:53.160770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:53.160770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:53.160770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:53.160770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:53.160770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:53.160770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:53.162055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:53.162055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:53.162055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:53.162055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:53.162055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:53.162055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:53.162055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:53.162055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:53.162055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:53.162055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:53.162055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 18:14:53.162055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:14:53.162055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 151/309] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:53.162055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8240b5e4-96eb-41f7-b750-e1bea1c35c6b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:53.177751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102f9e1-b395-4849-a034-f0bebbba3f21 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:53.226036 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae642e65-827e-4ad2-8622-8e6b4ed0dbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:53.226891 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae642e65-827e-4ad2-8622-8e6b4ed0dbf0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:53.227135 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae642e65-827e-4ad2-8622-8e6b4ed0dbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:53.227411 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae642e65-827e-4ad2-8622-8e6b4ed0dbf0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:53.228664 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae642e65-827e-4ad2-8622-8e6b4ed0dbf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:53.230211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae642e65-827e-4ad2-8622-8e6b4ed0dbf0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:53.232227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 159/310] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:14:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:53.232822 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae642e65-827e-4ad2-8622-8e6b4ed0dbf0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:53.322346 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8240b5e4-96eb-41f7-b750-e1bea1c35c6b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:53.380430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee77945-061c-443e-bc71-19fcc82ecd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:53.381423 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee77945-061c-443e-bc71-19fcc82ecd2f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:53.381807 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee77945-061c-443e-bc71-19fcc82ecd2f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:53.382832 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee77945-061c-443e-bc71-19fcc82ecd2f None admin] PATH_INFO: `/v3/users/8d64b4d970f04820a2faf68c5d8d349e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:53.383347 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae642e65-827e-4ad2-8622-8e6b4ed0dbf0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:53.402517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee77945-061c-443e-bc71-19fcc82ecd2f None admin] RBAC: Authorizing `identity:get_user(user_id=8d64b4d970f04820a2faf68c5d8d349e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:53.405349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee77945-061c-443e-bc71-19fcc82ecd2f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:53.413741 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 152/311] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:53 2022] GET /identity/v3/users/8d64b4d970f04820a2faf68c5d8d349e => generated 293 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:53.413741 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee77945-061c-443e-bc71-19fcc82ecd2f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:53.415552 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:53.437173 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144437af-fa97-4d8e-9c4e-3fa2aea810cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:53.438015 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144437af-fa97-4d8e-9c4e-3fa2aea810cc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:53.438351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144437af-fa97-4d8e-9c4e-3fa2aea810cc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:53.438695 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144437af-fa97-4d8e-9c4e-3fa2aea810cc None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:53.439983 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144437af-fa97-4d8e-9c4e-3fa2aea810cc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:53.440346 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144437af-fa97-4d8e-9c4e-3fa2aea810cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:53.442349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144437af-fa97-4d8e-9c4e-3fa2aea810cc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:53.455382 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 160/312] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:14:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:53.458651 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144437af-fa97-4d8e-9c4e-3fa2aea810cc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:53.482227 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:53.550684 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee77945-061c-443e-bc71-19fcc82ecd2f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:53.586926 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144437af-fa97-4d8e-9c4e-3fa2aea810cc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:53.611867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063b5cd-5f17-48d1-9df0-b28412218ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:53.612537 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063b5cd-5f17-48d1-9df0-b28412218ad0 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:53.612873 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063b5cd-5f17-48d1-9df0-b28412218ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:53.613143 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063b5cd-5f17-48d1-9df0-b28412218ad0 None admin] PATH_INFO: `/v3/groups` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:53.614083 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0063b5cd-5f17-48d1-9df0-b28412218ad0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:53.615558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0063b5cd-5f17-48d1-9df0-b28412218ad0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:53.639788 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1479332-738a-49d5-a079-103f8f48626c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:53.640974 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1479332-738a-49d5-a079-103f8f48626c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:53.641505 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1479332-738a-49d5-a079-103f8f48626c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:53.642070 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1479332-738a-49d5-a079-103f8f48626c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:53.643511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1479332-738a-49d5-a079-103f8f48626c None admin] RBAC: Adding query filter params (role.id=bdbac30ca21b4146bf95b3a053dff30a, user.id=8d64b4d970f04820a2faf68c5d8d349e, scope.system=all) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:53.644020 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1479332-738a-49d5-a079-103f8f48626c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:53.645981 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 153/313] 10.222.0.33 () {60 vars in 1134 bytes} [Thu Jun 30 18:14:53 2022] POST /identity/v3/groups => generated 229 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:14:53.646730 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1479332-738a-49d5-a079-103f8f48626c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:53.647957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063b5cd-5f17-48d1-9df0-b28412218ad0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:53.665959 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 161/314] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Jun 30 18:14:53 2022] GET /identity/v3/role_assignments?user.id=8d64b4d970f04820a2faf68c5d8d349e&scope.system=all&role.id=bdbac30ca21b4146bf95b3a053dff30a => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:53.843939 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063b5cd-5f17-48d1-9df0-b28412218ad0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:53.866976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:53.899586 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d7030-a417-4032-8eac-e25d29f64ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:53.900465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d7030-a417-4032-8eac-e25d29f64ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:53.900749 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d7030-a417-4032-8eac-e25d29f64ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:53.901026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d7030-a417-4032-8eac-e25d29f64ea5 None admin] PATH_INFO: `/v3/users/e98c10b29c0d4866b532931fd1855e0c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:53.902325 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d7030-a417-4032-8eac-e25d29f64ea5 None admin] RBAC: Authorizing `identity:get_user(user_id=e98c10b29c0d4866b532931fd1855e0c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:53.903794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d7030-a417-4032-8eac-e25d29f64ea5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:53.905661 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 154/315] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:53 2022] GET /identity/v3/users/e98c10b29c0d4866b532931fd1855e0c => generated 297 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:53.911169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1479332-738a-49d5-a079-103f8f48626c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:53.967782 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:54.031333 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1479332-738a-49d5-a079-103f8f48626c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:54.069754 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940140e-a049-45b6-b81a-12653b630c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:54.070901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940140e-a049-45b6-b81a-12653b630c06 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:54.071419 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940140e-a049-45b6-b81a-12653b630c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:54.071916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940140e-a049-45b6-b81a-12653b630c06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:54.167952 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940140e-a049-45b6-b81a-12653b630c06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:54.170110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940140e-a049-45b6-b81a-12653b630c06 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:54.259792 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:54.324158 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 162/316] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:14:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:14:54.329701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d7030-a417-4032-8eac-e25d29f64ea5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:54.443526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d7030-a417-4032-8eac-e25d29f64ea5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:54.485986 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e66f53-acbd-41a1-be30-55ac5b555355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:54.487114 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e66f53-acbd-41a1-be30-55ac5b555355 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:54.487584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e66f53-acbd-41a1-be30-55ac5b555355 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:54.488013 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e66f53-acbd-41a1-be30-55ac5b555355 None admin] PATH_INFO: `/v3/projects/9beede2985d54385bc4532fb19a265a5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:54.489462 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e66f53-acbd-41a1-be30-55ac5b555355 None admin] RBAC: Authorizing `identity:get_project(project_id=9beede2985d54385bc4532fb19a265a5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:54.491275 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e66f53-acbd-41a1-be30-55ac5b555355 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:54.493665 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 155/317] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:14:54 2022] GET /identity/v3/projects/9beede2985d54385bc4532fb19a265a5 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:54.502781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940140e-a049-45b6-b81a-12653b630c06 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:54.636424 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940140e-a049-45b6-b81a-12653b630c06 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:54.674906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7187c-a5eb-4fed-a3b9-4f05f0568e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:54.675577 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7187c-a5eb-4fed-a3b9-4f05f0568e41 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:54.675864 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7187c-a5eb-4fed-a3b9-4f05f0568e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:54.676146 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7187c-a5eb-4fed-a3b9-4f05f0568e41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:54.676988 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7187c-a5eb-4fed-a3b9-4f05f0568e41 None admin] RBAC: Adding query filter params (role.id=bdbac30ca21b4146bf95b3a053dff30a, scope.project.id=9beede2985d54385bc4532fb19a265a5, user.id=e98c10b29c0d4866b532931fd1855e0c) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:54.677303 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7187c-a5eb-4fed-a3b9-4f05f0568e41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:54.679346 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7187c-a5eb-4fed-a3b9-4f05f0568e41 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:54.685793 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 163/318] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:14:54 2022] GET /identity/v3/role_assignments?user.id=e98c10b29c0d4866b532931fd1855e0c&scope.project.id=9beede2985d54385bc4532fb19a265a5&role.id=bdbac30ca21b4146bf95b3a053dff30a => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:54.979901 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e730b20-226a-40dc-9d49-6468a39636c3 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:54.980540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e730b20-226a-40dc-9d49-6468a39636c3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:54.980540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e730b20-226a-40dc-9d49-6468a39636c3 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:54.982154 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 156/319] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:14:54.987848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70efd1-24b5-4981-b3d9-fed86a4e02b6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:54.988414 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70efd1-24b5-4981-b3d9-fed86a4e02b6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:54.988414 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70efd1-24b5-4981-b3d9-fed86a4e02b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:55.010098 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-eb70efd1-24b5-4981-b3d9-fed86a4e02b6 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:55.011433 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-eb70efd1-24b5-4981-b3d9-fed86a4e02b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:55.020506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:55.102774 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 164/320] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 18:14:55.127637 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e730b20-226a-40dc-9d49-6468a39636c3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:55.188253 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:55.259413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e730b20-226a-40dc-9d49-6468a39636c3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:55.306092 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47aaa92-71fe-4c1f-8567-957c319ea58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:55.307531 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47aaa92-71fe-4c1f-8567-957c319ea58b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:55.308020 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47aaa92-71fe-4c1f-8567-957c319ea58b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:55.308548 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47aaa92-71fe-4c1f-8567-957c319ea58b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:55.326656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47aaa92-71fe-4c1f-8567-957c319ea58b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:55.341727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f47aaa92-71fe-4c1f-8567-957c319ea58b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:14:55.341727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:55.341727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:55.341727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:55.341727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:55.341727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:55.341727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:55.341727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:55.341727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:55.341727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:55.341727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:55.341727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:55.341727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:55.341727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:55.341727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:55.341727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:55.343615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:55.343615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:55.343615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:55.343615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:55.343615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:55.343615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:55.343615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:55.343615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:55.343615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:55.343615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:55.343615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:55.343615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:55.343615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:55.343615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:55.343615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:55.343615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:55.345026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:55.345026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:55.345026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:55.345026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:55.345026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:55.345026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:55.345026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:55.345026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:55.345026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:55.345026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:55.345026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:14:55.345026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:14:55.347159 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 157/321] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:14:55.353648 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70efd1-24b5-4981-b3d9-fed86a4e02b6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:55.517843 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70efd1-24b5-4981-b3d9-fed86a4e02b6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:55.569533 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4cb45-9c38-4975-a2e5-4546ce834b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:55.570250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4cb45-9c38-4975-a2e5-4546ce834b75 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:55.570541 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4cb45-9c38-4975-a2e5-4546ce834b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:55.570814 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4cb45-9c38-4975-a2e5-4546ce834b75 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:55.571573 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4cb45-9c38-4975-a2e5-4546ce834b75 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:55.571879 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4cb45-9c38-4975-a2e5-4546ce834b75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:55.573267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4cb45-9c38-4975-a2e5-4546ce834b75 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:55.584192 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 165/322] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:14:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:55.590036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47aaa92-71fe-4c1f-8567-957c319ea58b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:55.715727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47aaa92-71fe-4c1f-8567-957c319ea58b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:55.726649 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:55.759599 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ef5b5-8939-4111-93dd-eb0aa2938693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:55.760738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ef5b5-8939-4111-93dd-eb0aa2938693 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:55.761023 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ef5b5-8939-4111-93dd-eb0aa2938693 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:55.761415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ef5b5-8939-4111-93dd-eb0aa2938693 None admin] PATH_INFO: `/v3/users/24310fb42a0947cd81738be84d5f53ef` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:55.763017 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ef5b5-8939-4111-93dd-eb0aa2938693 None admin] RBAC: Authorizing `identity:get_user(user_id=24310fb42a0947cd81738be84d5f53ef)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:55.764510 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ef5b5-8939-4111-93dd-eb0aa2938693 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:55.766511 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 158/323] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:55 2022] GET /identity/v3/users/24310fb42a0947cd81738be84d5f53ef => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:14:55.772969 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4cb45-9c38-4975-a2e5-4546ce834b75 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:55.778232 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:55.934309 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4c70b-73c0-48cb-9c56-33675542b17e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:55.934753 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4c70b-73c0-48cb-9c56-33675542b17e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:55.935102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4cb45-9c38-4975-a2e5-4546ce834b75 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:55.935252 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4c70b-73c0-48cb-9c56-33675542b17e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:55.937629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 159/324] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:14:55.945907 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff39a64-ee7e-45ea-92d7-a0d9b0dc9030 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:55.946380 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff39a64-ee7e-45ea-92d7-a0d9b0dc9030 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:55.946651 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff39a64-ee7e-45ea-92d7-a0d9b0dc9030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:55.970004 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-dff39a64-ee7e-45ea-92d7-a0d9b0dc9030 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:55.971679 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dff39a64-ee7e-45ea-92d7-a0d9b0dc9030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:55.982120 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23855b9d-e05c-407e-8115-235e6cb3f430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:55.982893 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23855b9d-e05c-407e-8115-235e6cb3f430 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:55.983083 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23855b9d-e05c-407e-8115-235e6cb3f430 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:55.983357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23855b9d-e05c-407e-8115-235e6cb3f430 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:55.984331 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23855b9d-e05c-407e-8115-235e6cb3f430 None admin] RBAC: Adding query filter params (role.id=3fbb71ae2de04e78b68d6d6f407ef09c, user.id=24310fb42a0947cd81738be84d5f53ef, scope.system=all) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:55.984531 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23855b9d-e05c-407e-8115-235e6cb3f430 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:55.986089 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23855b9d-e05c-407e-8115-235e6cb3f430 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:56.001093 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 166/325] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Jun 30 18:14:55 2022] GET /identity/v3/role_assignments?user.id=24310fb42a0947cd81738be84d5f53ef&scope.system=all&role.id=3fbb71ae2de04e78b68d6d6f407ef09c => generated 510 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:56.035223 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9223b6-5ab0-4501-b250-4b767f342ad0 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:56.035522 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9223b6-5ab0-4501-b250-4b767f342ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:56.035810 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9223b6-5ab0-4501-b250-4b767f342ad0 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:56.037487 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 167/326] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:14:56.044758 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4a28f-f3d1-41d5-9fc5-47d94ce5455d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:56.045161 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4a28f-f3d1-41d5-9fc5-47d94ce5455d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:56.045379 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4a28f-f3d1-41d5-9fc5-47d94ce5455d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:56.064934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 160/327] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 18:14:56.072171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-81a4a28f-f3d1-41d5-9fc5-47d94ce5455d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:56.073423 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-81a4a28f-f3d1-41d5-9fc5-47d94ce5455d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:56.091566 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff39a64-ee7e-45ea-92d7-a0d9b0dc9030 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:56.150184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:56.164712 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 168/328] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:56.189068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4a28f-f3d1-41d5-9fc5-47d94ce5455d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:56.276784 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff39a64-ee7e-45ea-92d7-a0d9b0dc9030 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:56.288434 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:56.326565 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef9773-47fb-43fd-b7d1-0e5a993ee429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:56.327385 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef9773-47fb-43fd-b7d1-0e5a993ee429 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:56.327674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef9773-47fb-43fd-b7d1-0e5a993ee429 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:56.327953 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef9773-47fb-43fd-b7d1-0e5a993ee429 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:56.340603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef9773-47fb-43fd-b7d1-0e5a993ee429 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:56.354638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-45ef9773-47fb-43fd-b7d1-0e5a993ee429 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 18:14:56.354638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:56.354638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:56.354638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:56.354638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:56.354638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:56.354638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:56.354638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:56.354638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:56.354638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:56.354638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:56.354638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:56.354638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:56.354638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:56.354638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:56.354638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:56.355809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:56.355809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:56.355809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:56.355809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:56.355809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:56.355809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:56.355809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:56.355809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:56.355809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:56.355809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:56.355809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:56.355809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:56.355809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:56.355809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:56.355809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:56.355809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:56.356657 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:56.356657 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:56.356657 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:56.356657 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:56.356657 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:56.356657 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:56.356657 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:56.356657 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:56.356657 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:56.356657 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:56.356657 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 18:14:56.356657 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:14:56.356657 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 161/329] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:14:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:14:56.363279 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef9773-47fb-43fd-b7d1-0e5a993ee429 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:56.368609 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4a28f-f3d1-41d5-9fc5-47d94ce5455d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:56.413881 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3268d23-b763-4827-9b5d-ba2777784a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:56.414707 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3268d23-b763-4827-9b5d-ba2777784a67 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:56.414991 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3268d23-b763-4827-9b5d-ba2777784a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:56.415267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3268d23-b763-4827-9b5d-ba2777784a67 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:56.416532 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3268d23-b763-4827-9b5d-ba2777784a67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:56.418123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3268d23-b763-4827-9b5d-ba2777784a67 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:56.420144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 169/330] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:14:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:56.426005 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3268d23-b763-4827-9b5d-ba2777784a67 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:56.488765 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef9773-47fb-43fd-b7d1-0e5a993ee429 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:56.495172 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:56.532701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a14684-a3bc-48bd-a76f-b7d2f70d3468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:56.533714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a14684-a3bc-48bd-a76f-b7d2f70d3468 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:56.534229 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a14684-a3bc-48bd-a76f-b7d2f70d3468 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:56.534390 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a14684-a3bc-48bd-a76f-b7d2f70d3468 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:56.535526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a14684-a3bc-48bd-a76f-b7d2f70d3468 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:14:56.535919 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a14684-a3bc-48bd-a76f-b7d2f70d3468 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:56.537609 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a14684-a3bc-48bd-a76f-b7d2f70d3468 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:56.555847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 162/331] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:14:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:56.562438 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a14684-a3bc-48bd-a76f-b7d2f70d3468 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:56.565542 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3268d23-b763-4827-9b5d-ba2777784a67 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:56.609473 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:56.635397 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6824b1-7e0b-4296-abed-a9c084e53dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:56.636274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6824b1-7e0b-4296-abed-a9c084e53dbf None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:56.636647 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6824b1-7e0b-4296-abed-a9c084e53dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:56.637020 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6824b1-7e0b-4296-abed-a9c084e53dbf None admin] PATH_INFO: `/v3/groups` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:56.637877 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6824b1-7e0b-4296-abed-a9c084e53dbf None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:56.639676 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6824b1-7e0b-4296-abed-a9c084e53dbf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:56.659013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 170/332] 10.222.0.33 () {60 vars in 1134 bytes} [Thu Jun 30 18:14:56 2022] POST /identity/v3/groups => generated 226 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:14:56.703008 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a14684-a3bc-48bd-a76f-b7d2f70d3468 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:56.745211 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5780a9f-4b3e-4f94-98db-126c3aef77b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:56.746149 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5780a9f-4b3e-4f94-98db-126c3aef77b2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:56.746451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5780a9f-4b3e-4f94-98db-126c3aef77b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:56.746721 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5780a9f-4b3e-4f94-98db-126c3aef77b2 None admin] PATH_INFO: `/v3/users/8d64b4d970f04820a2faf68c5d8d349e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:56.748114 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5780a9f-4b3e-4f94-98db-126c3aef77b2 None admin] RBAC: Authorizing `identity:get_user(user_id=8d64b4d970f04820a2faf68c5d8d349e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:56.749671 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5780a9f-4b3e-4f94-98db-126c3aef77b2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:56.751461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 163/333] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:14:56 2022] GET /identity/v3/users/8d64b4d970f04820a2faf68c5d8d349e => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:14:56.757433 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6824b1-7e0b-4296-abed-a9c084e53dbf None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:56.872601 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6824b1-7e0b-4296-abed-a9c084e53dbf None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:56.899762 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:56.921814 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0c730-f12e-4d54-ae99-b4f7503b91a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:56.923039 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0c730-f12e-4d54-ae99-b4f7503b91a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:56.923364 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0c730-f12e-4d54-ae99-b4f7503b91a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:56.923684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0c730-f12e-4d54-ae99-b4f7503b91a5 None admin] PATH_INFO: `/v3/system/users/8d64b4d970f04820a2faf68c5d8d349e/roles/bdbac30ca21b4146bf95b3a053dff30a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:56.925343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b0c730-f12e-4d54-ae99-b4f7503b91a5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8d64b4d970f04820a2faf68c5d8d349e, role_id=bdbac30ca21b4146bf95b3a053dff30a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:56.926834 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b0c730-f12e-4d54-ae99-b4f7503b91a5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:56.935810 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 171/334] 10.222.0.33 () {58 vars in 1334 bytes} [Thu Jun 30 18:14:56 2022] PUT /identity/v3/system/users/8d64b4d970f04820a2faf68c5d8d349e/roles/bdbac30ca21b4146bf95b3a053dff30a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:14:59.373901 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c462c66-a2e0-4941-9c4d-1bb2356407ca None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:59.374976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c462c66-a2e0-4941-9c4d-1bb2356407ca None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:59.375488 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c462c66-a2e0-4941-9c4d-1bb2356407ca None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:59.377196 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 164/335] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:14:59.385543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88821288-d2f3-4230-98c4-817c794e7453 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:59.386552 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88821288-d2f3-4230-98c4-817c794e7453 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:59.387054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88821288-d2f3-4230-98c4-817c794e7453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:59.410953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-88821288-d2f3-4230-98c4-817c794e7453 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:59.412581 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-88821288-d2f3-4230-98c4-817c794e7453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:59.508382 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 172/336] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:14:59.578315 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c462c66-a2e0-4941-9c4d-1bb2356407ca None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:59.802406 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92571a-3794-4b4f-8520-d321a02d8e47 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:59.802997 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92571a-3794-4b4f-8520-d321a02d8e47 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:59.803554 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92571a-3794-4b4f-8520-d321a02d8e47 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:59.808054 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:59.808470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 173/337] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:14:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:14:59.816822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c462c66-a2e0-4941-9c4d-1bb2356407ca None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:59.817045 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d34d08-66f4-4693-9845-f5c02eba5be1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:59.817605 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d34d08-66f4-4693-9845-f5c02eba5be1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:59.818111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d34d08-66f4-4693-9845-f5c02eba5be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:59.854650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-70d34d08-66f4-4693-9845-f5c02eba5be1 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:14:59.856267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-70d34d08-66f4-4693-9845-f5c02eba5be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:59.896415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99444b3-a575-4ff1-bebb-7487ec2e9b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:59.901618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99444b3-a575-4ff1-bebb-7487ec2e9b96 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:59.901618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99444b3-a575-4ff1-bebb-7487ec2e9b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:59.901618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99444b3-a575-4ff1-bebb-7487ec2e9b96 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:59.921287 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99444b3-a575-4ff1-bebb-7487ec2e9b96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:14:59.932659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d99444b3-a575-4ff1-bebb-7487ec2e9b96 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:14:59.932659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:14:59.932659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:14:59.932659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:14:59.932659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:14:59.932659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:14:59.932659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:14:59.932659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:14:59.932659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:14:59.932659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:14:59.932659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:14:59.932659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:14:59.932659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:14:59.932659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:14:59.932659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:14:59.932659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:14:59.933645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:14:59.933645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:14:59.933645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:14:59.933645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:14:59.933645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:14:59.933645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:14:59.933645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:14:59.933645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:14:59.933645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:14:59.933645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:14:59.933645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:14:59.933645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:14:59.933645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:14:59.933645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:14:59.933645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:14:59.933645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:14:59.934918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:14:59.934918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:14:59.934918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:14:59.934918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:14:59.934918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:14:59.934918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:14:59.934918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:14:59.934918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:14:59.934918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:14:59.934918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:14:59.934918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:14:59.934918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:14:59.934918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 165/338] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 30 18:14:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:14:59.940855 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99444b3-a575-4ff1-bebb-7487ec2e9b96 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:00.024029 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 174/339] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 18:15:00.025870 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448287c2-1287-492c-b18c-79ca2ae6185d None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:00.026409 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448287c2-1287-492c-b18c-79ca2ae6185d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:00.026894 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448287c2-1287-492c-b18c-79ca2ae6185d None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:00.029527 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 175/340] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:00.035796 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9515e2d-c2b9-46c7-b9f1-b70ec04bdc68 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:00.036600 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9515e2d-c2b9-46c7-b9f1-b70ec04bdc68 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:00.037111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9515e2d-c2b9-46c7-b9f1-b70ec04bdc68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:00.059745 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d9515e2d-c2b9-46c7-b9f1-b70ec04bdc68 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:00.061932 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d9515e2d-c2b9-46c7-b9f1-b70ec04bdc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:00.140406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99444b3-a575-4ff1-bebb-7487ec2e9b96 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:00.144233 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:00.184441 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5bd9e-22ea-4648-800c-9492b193d319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:00.185710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5bd9e-22ea-4648-800c-9492b193d319 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:00.186230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5bd9e-22ea-4648-800c-9492b193d319 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:00.186730 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5bd9e-22ea-4648-800c-9492b193d319 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:00.187728 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5bd9e-22ea-4648-800c-9492b193d319 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:00.188273 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5bd9e-22ea-4648-800c-9492b193d319 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:00.189996 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5bd9e-22ea-4648-800c-9492b193d319 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:00.200898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 166/341] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:14:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:15:00.202852 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7214c4f-2ebe-4d87-a1fa-4aee8aa777fc None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:00.203497 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7214c4f-2ebe-4d87-a1fa-4aee8aa777fc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:00.204019 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7214c4f-2ebe-4d87-a1fa-4aee8aa777fc None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:00.210332 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 167/342] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:00.212100 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7214c4f-2ebe-4d87-a1fa-4aee8aa777fc None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:00.213510 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 176/343] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 18:15:00.214779 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9515e2d-c2b9-46c7-b9f1-b70ec04bdc68 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:00.262884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:00.417078 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7214c4f-2ebe-4d87-a1fa-4aee8aa777fc None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:00.418095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9515e2d-c2b9-46c7-b9f1-b70ec04bdc68 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:00.466256 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50748cec-a62f-427c-b009-25662e3f5f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:00.467976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50748cec-a62f-427c-b009-25662e3f5f17 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:00.468995 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50748cec-a62f-427c-b009-25662e3f5f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:00.469289 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50748cec-a62f-427c-b009-25662e3f5f17 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:00.473318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674154ed-8348-49b1-810a-19c7c9445238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:00.474491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674154ed-8348-49b1-810a-19c7c9445238 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:00.475840 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674154ed-8348-49b1-810a-19c7c9445238 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:00.476454 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674154ed-8348-49b1-810a-19c7c9445238 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:00.484842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50748cec-a62f-427c-b009-25662e3f5f17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:00.494301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-50748cec-a62f-427c-b009-25662e3f5f17 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:15:00.494301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:00.494301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:00.494301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:00.494301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:00.494301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:00.494301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:00.494301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:00.494301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:00.494301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:00.494301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:00.494301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:00.494301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:00.494301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:00.494301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:00.494301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:00.495517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:00.495517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:00.495517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:00.495517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:00.495517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:00.495517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:00.495517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:00.495517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:00.495517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:00.495517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:00.495517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:00.495517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:00.495517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:00.495517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:00.495517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:00.495517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:00.496428 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:00.496428 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:00.496428 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:00.496428 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:00.496428 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:00.496428 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:00.496428 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:00.496428 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:00.496428 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:00.496428 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:00.496428 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:15:00.496428 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:00.499965 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 168/344] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:15:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:00.501776 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425e6da-0ff3-4568-8f81-7c47182ee03d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:00.502281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425e6da-0ff3-4568-8f81-7c47182ee03d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:00.502742 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425e6da-0ff3-4568-8f81-7c47182ee03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:00.525125 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9425e6da-0ff3-4568-8f81-7c47182ee03d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:00.526922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9425e6da-0ff3-4568-8f81-7c47182ee03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:00.530366 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:00.566757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674154ed-8348-49b1-810a-19c7c9445238 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:00.568478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674154ed-8348-49b1-810a-19c7c9445238 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:00.632580 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 169/345] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:00.636061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9425e6da-0ff3-4568-8f81-7c47182ee03d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:00.652493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:00.724766 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 177/346] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:15:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:15:00.726329 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674154ed-8348-49b1-810a-19c7c9445238 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:00.783881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9425e6da-0ff3-4568-8f81-7c47182ee03d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:00.813698 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:00.824874 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e50d5-3165-4efd-9574-f3a2082917c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:00.825765 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e50d5-3165-4efd-9574-f3a2082917c7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:00.826123 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e50d5-3165-4efd-9574-f3a2082917c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:00.826468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e50d5-3165-4efd-9574-f3a2082917c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:00.838766 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e50d5-3165-4efd-9574-f3a2082917c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:00.846090 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674154ed-8348-49b1-810a-19c7c9445238 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:00.847387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e34e50d5-3165-4efd-9574-f3a2082917c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:15:00.847387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:00.847387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:00.847387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:00.847387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:00.847387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:00.847387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:00.847387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:00.847387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:00.847387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:00.847387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:00.847387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:00.847387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:00.847387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:00.847387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:00.847387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:00.848276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:00.848276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:00.848276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:00.848276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:00.848276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:00.848276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:00.848276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:00.848276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:00.848276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:00.848276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:00.848276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:00.848276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:00.848276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:00.848276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:00.848276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:00.848276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:00.849089 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:00.849089 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:00.849089 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:00.849089 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:00.849089 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:00.849089 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:00.849089 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:00.849089 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:00.849089 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:00.849089 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:00.849089 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:15:00.849089 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:00.849990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 170/347] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:15:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:00.854755 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e50d5-3165-4efd-9574-f3a2082917c7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:00.889066 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7285d6-60f7-42b2-a184-5676a43ecdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:00.890089 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7285d6-60f7-42b2-a184-5676a43ecdbc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:00.890580 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7285d6-60f7-42b2-a184-5676a43ecdbc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:00.891057 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7285d6-60f7-42b2-a184-5676a43ecdbc None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:00.891997 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7285d6-60f7-42b2-a184-5676a43ecdbc None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:00.892504 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7285d6-60f7-42b2-a184-5676a43ecdbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:00.894174 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7285d6-60f7-42b2-a184-5676a43ecdbc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:00.904721 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 178/348] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:15:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:00.906020 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7285d6-60f7-42b2-a184-5676a43ecdbc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:00.938395 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:00.981918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e50d5-3165-4efd-9574-f3a2082917c7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:01.042369 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678e2be-5f5a-4684-971e-5458666ef506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:01.043303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678e2be-5f5a-4684-971e-5458666ef506 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:01.043500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678e2be-5f5a-4684-971e-5458666ef506 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:01.044005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678e2be-5f5a-4684-971e-5458666ef506 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:01.059704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7285d6-60f7-42b2-a184-5676a43ecdbc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:01.072226 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9678e2be-5f5a-4684-971e-5458666ef506 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:01.082209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-9678e2be-5f5a-4684-971e-5458666ef506 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 18:15:01.082209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:01.082209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:01.082209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:01.082209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:01.082209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:01.082209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:01.082209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:01.082209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:01.082209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:01.082209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:01.082209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:01.082209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:01.082209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:01.082209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:01.082209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:01.083529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:01.083529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:01.083529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:01.083529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:01.083529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:01.083529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:01.083529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:01.083529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:01.083529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:01.083529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:01.083529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:01.083529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:01.083529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:01.083529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:01.083529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:01.083529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:01.085911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:01.085911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:01.085911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:01.085911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:01.085911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:01.085911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:01.085911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:01.085911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:01.085911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:01.085911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:01.085911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 18:15:01.085911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:01.085911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 171/349] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:15:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:01.091155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678e2be-5f5a-4684-971e-5458666ef506 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:01.111946 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e74a3-e08e-4cfe-8cb9-4265078a3c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:01.113866 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e74a3-e08e-4cfe-8cb9-4265078a3c69 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:01.114287 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e74a3-e08e-4cfe-8cb9-4265078a3c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:01.114628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e74a3-e08e-4cfe-8cb9-4265078a3c69 None admin] PATH_INFO: `/v3/projects/d5962edfa0c94d88a1056c1ed5e05058` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:01.116407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e74a3-e08e-4cfe-8cb9-4265078a3c69 None admin] RBAC: Authorizing `identity:get_project(project_id=d5962edfa0c94d88a1056c1ed5e05058)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:01.117941 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e74a3-e08e-4cfe-8cb9-4265078a3c69 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:01.119983 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 179/350] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:15:00 2022] GET /identity/v3/projects/d5962edfa0c94d88a1056c1ed5e05058 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:01.120568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e74a3-e08e-4cfe-8cb9-4265078a3c69 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:01.124757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:01.283067 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678e2be-5f5a-4684-971e-5458666ef506 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:01.286350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e74a3-e08e-4cfe-8cb9-4265078a3c69 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:01.326661 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:01.341896 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28017b1-f9f9-4277-9c18-d84d3ae1c53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:01.343029 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28017b1-f9f9-4277-9c18-d84d3ae1c53c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:01.343487 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28017b1-f9f9-4277-9c18-d84d3ae1c53c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:01.343954 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28017b1-f9f9-4277-9c18-d84d3ae1c53c None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:01.344891 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f955aca-df4a-43e6-b169-8fd9c5084c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:01.345351 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28017b1-f9f9-4277-9c18-d84d3ae1c53c None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:01.345682 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f955aca-df4a-43e6-b169-8fd9c5084c27 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:01.345968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f955aca-df4a-43e6-b169-8fd9c5084c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:01.346077 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28017b1-f9f9-4277-9c18-d84d3ae1c53c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:01.346347 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f955aca-df4a-43e6-b169-8fd9c5084c27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:01.348480 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28017b1-f9f9-4277-9c18-d84d3ae1c53c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:01.368263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 172/351] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:15:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:01.371673 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b458705-d827-4cf0-b81d-cda54e740353 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:01.372164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b458705-d827-4cf0-b81d-cda54e740353 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:01.373762 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b458705-d827-4cf0-b81d-cda54e740353 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:01.376483 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 173/352] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:01.378146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b458705-d827-4cf0-b81d-cda54e740353 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:01.448824 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f955aca-df4a-43e6-b169-8fd9c5084c27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:01.450404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f955aca-df4a-43e6-b169-8fd9c5084c27 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:01.454707 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:01.592130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b458705-d827-4cf0-b81d-cda54e740353 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:01.638275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 180/353] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:15:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:15:01.640701 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f955aca-df4a-43e6-b169-8fd9c5084c27 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:01.666676 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263fbe6-dfc0-42b0-9b4f-f8bd910964ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:01.667828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263fbe6-dfc0-42b0-9b4f-f8bd910964ef None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:01.668285 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263fbe6-dfc0-42b0-9b4f-f8bd910964ef None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:01.669018 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263fbe6-dfc0-42b0-9b4f-f8bd910964ef None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:01.672029 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263fbe6-dfc0-42b0-9b4f-f8bd910964ef None admin] RBAC: Adding query filter params (name=reader) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:01.672594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263fbe6-dfc0-42b0-9b4f-f8bd910964ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:01.675321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263fbe6-dfc0-42b0-9b4f-f8bd910964ef None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:01.692885 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 174/354] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:15:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:01.697936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263fbe6-dfc0-42b0-9b4f-f8bd910964ef None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:01.826657 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:01.846199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f955aca-df4a-43e6-b169-8fd9c5084c27 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:01.850089 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:01.899155 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62b3c6-cb82-408b-9b0f-309e2bda06c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:01.900110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62b3c6-cb82-408b-9b0f-309e2bda06c9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:01.900417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62b3c6-cb82-408b-9b0f-309e2bda06c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:01.900684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62b3c6-cb82-408b-9b0f-309e2bda06c9 None admin] PATH_INFO: `/v3/groups/bb43d54bfa084cd59d6ce44ca23cdb96` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:01.900948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263fbe6-dfc0-42b0-9b4f-f8bd910964ef None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:01.910085 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b62b3c6-cb82-408b-9b0f-309e2bda06c9 None admin] RBAC: Authorizing `identity:get_group(group_id=bb43d54bfa084cd59d6ce44ca23cdb96)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:01.911741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b62b3c6-cb82-408b-9b0f-309e2bda06c9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:01.915084 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 181/355] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 30 18:15:01 2022] GET /identity/v3/groups/bb43d54bfa084cd59d6ce44ca23cdb96 => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:01.917784 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e38a92-e5b5-44e0-a541-57435068a1ad None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:01.917784 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e38a92-e5b5-44e0-a541-57435068a1ad None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:01.917784 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e38a92-e5b5-44e0-a541-57435068a1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:01.946987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a7e38a92-e5b5-44e0-a541-57435068a1ad None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:01.948243 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a7e38a92-e5b5-44e0-a541-57435068a1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:01.950670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c650557-1472-461a-89e0-713e85353a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:01.951336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c650557-1472-461a-89e0-713e85353a45 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:01.951618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c650557-1472-461a-89e0-713e85353a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:01.951893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c650557-1472-461a-89e0-713e85353a45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:02.049125 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c650557-1472-461a-89e0-713e85353a45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:02.051923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c650557-1472-461a-89e0-713e85353a45 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:02.073764 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 182/356] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:02.078830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e38a92-e5b5-44e0-a541-57435068a1ad None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:02.128399 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:02.214541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 175/357] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:15:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:15:02.215424 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c650557-1472-461a-89e0-713e85353a45 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:02.247794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:02.254071 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e38a92-e5b5-44e0-a541-57435068a1ad None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:02.320490 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b279eec-c6bb-488e-affd-a59a41778cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:02.322770 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b279eec-c6bb-488e-affd-a59a41778cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:02.322770 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b279eec-c6bb-488e-affd-a59a41778cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:02.322770 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b279eec-c6bb-488e-affd-a59a41778cd0 None admin] PATH_INFO: `/v3/projects/d5962edfa0c94d88a1056c1ed5e05058` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:02.324939 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b279eec-c6bb-488e-affd-a59a41778cd0 None admin] RBAC: Authorizing `identity:get_project(project_id=d5962edfa0c94d88a1056c1ed5e05058)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:02.327764 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b279eec-c6bb-488e-affd-a59a41778cd0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:02.331369 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 183/358] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:15:02 2022] GET /identity/v3/projects/d5962edfa0c94d88a1056c1ed5e05058 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:02.333640 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992dd6c-491f-4ca2-bed8-6bae23b8aafc None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:02.334148 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992dd6c-491f-4ca2-bed8-6bae23b8aafc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:02.334806 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992dd6c-491f-4ca2-bed8-6bae23b8aafc None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:02.338346 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 184/359] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:02.339844 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992dd6c-491f-4ca2-bed8-6bae23b8aafc None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:02.394806 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c650557-1472-461a-89e0-713e85353a45 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:02.453571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e99cc-a71d-4a03-9dd2-b50541660a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:02.454983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e99cc-a71d-4a03-9dd2-b50541660a12 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:02.454983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e99cc-a71d-4a03-9dd2-b50541660a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:02.455279 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e99cc-a71d-4a03-9dd2-b50541660a12 None admin] PATH_INFO: `/v3/users/8d64b4d970f04820a2faf68c5d8d349e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:02.458344 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e99cc-a71d-4a03-9dd2-b50541660a12 None admin] RBAC: Authorizing `identity:get_user(user_id=8d64b4d970f04820a2faf68c5d8d349e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:02.458344 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e99cc-a71d-4a03-9dd2-b50541660a12 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:02.461169 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 176/360] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:15:02 2022] GET /identity/v3/users/8d64b4d970f04820a2faf68c5d8d349e => generated 293 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:02.462204 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e99cc-a71d-4a03-9dd2-b50541660a12 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:02.493049 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:02.503809 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992dd6c-491f-4ca2-bed8-6bae23b8aafc None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:02.568487 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a024b-042d-4f60-a1ad-5872b3f3ffb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:02.570054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a024b-042d-4f60-a1ad-5872b3f3ffb3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:02.570346 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a024b-042d-4f60-a1ad-5872b3f3ffb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:02.570627 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a024b-042d-4f60-a1ad-5872b3f3ffb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:02.571499 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0a024b-042d-4f60-a1ad-5872b3f3ffb3 None admin] RBAC: Adding query filter params (group.id=bb43d54bfa084cd59d6ce44ca23cdb96, role.id=d5f874a3d78f4ee0ade318e1bbebf7ec, scope.project.id=d5962edfa0c94d88a1056c1ed5e05058) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:02.571813 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0a024b-042d-4f60-a1ad-5872b3f3ffb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:02.573328 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0a024b-042d-4f60-a1ad-5872b3f3ffb3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:02.579853 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 185/361] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 30 18:15:02 2022] GET /identity/v3/role_assignments?group.id=bb43d54bfa084cd59d6ce44ca23cdb96&scope.project.id=d5962edfa0c94d88a1056c1ed5e05058&role.id=d5f874a3d78f4ee0ade318e1bbebf7ec => generated 260 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:02.580729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a024b-042d-4f60-a1ad-5872b3f3ffb3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:02.633952 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:02.634916 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e99cc-a71d-4a03-9dd2-b50541660a12 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:02.691570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ebf75-5c96-41d2-81b1-05fc46cc87d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:02.692415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ebf75-5c96-41d2-81b1-05fc46cc87d8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:02.693061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ebf75-5c96-41d2-81b1-05fc46cc87d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:02.693433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ebf75-5c96-41d2-81b1-05fc46cc87d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:02.718272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ebf75-5c96-41d2-81b1-05fc46cc87d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:02.729144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-538ebf75-5c96-41d2-81b1-05fc46cc87d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:15:02.729144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:02.729144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:02.729144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:02.729144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:02.729144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:02.729144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:02.729144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:02.729144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:02.729144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:02.729144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:02.729144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:02.729144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:02.729144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:02.729144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:02.729144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:02.730911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:02.730911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:02.730911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:02.730911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:02.730911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:02.730911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:02.730911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:02.730911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:02.730911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:02.730911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:02.730911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:02.730911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:02.730911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:02.730911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:02.730911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:02.730911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:02.732461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:02.732461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:02.732461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:02.732461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:02.732461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:02.732461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:02.732461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:02.732461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:02.732461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:02.732461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:02.732461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:15:02.732461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:02.732461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 177/362] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 30 18:15:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:02.732461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ebf75-5c96-41d2-81b1-05fc46cc87d8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:02.778908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a024b-042d-4f60-a1ad-5872b3f3ffb3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:02.835667 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959453b5-c2b4-441b-bacb-98132411aa39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:02.836686 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959453b5-c2b4-441b-bacb-98132411aa39 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:02.836987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959453b5-c2b4-441b-bacb-98132411aa39 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:02.837271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959453b5-c2b4-441b-bacb-98132411aa39 None admin] PATH_INFO: `/v3/projects/9beede2985d54385bc4532fb19a265a5` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:02.838768 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959453b5-c2b4-441b-bacb-98132411aa39 None admin] RBAC: Authorizing `identity:get_project(project_id=9beede2985d54385bc4532fb19a265a5)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:02.840272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959453b5-c2b4-441b-bacb-98132411aa39 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:02.843203 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 186/363] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:15:02 2022] GET /identity/v3/projects/9beede2985d54385bc4532fb19a265a5 => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:02.843843 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe2847-e624-4313-887b-f6052d554eed None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:02.844137 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe2847-e624-4313-887b-f6052d554eed None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:02.844422 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe2847-e624-4313-887b-f6052d554eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:02.868351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b7fe2847-e624-4313-887b-f6052d554eed None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:02.869613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b7fe2847-e624-4313-887b-f6052d554eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:02.871332 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:02.902985 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ebf75-5c96-41d2-81b1-05fc46cc87d8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:02.953737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552dd821-250d-47f2-946b-6624cd91cd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:02.954944 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552dd821-250d-47f2-946b-6624cd91cd08 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:02.955421 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552dd821-250d-47f2-946b-6624cd91cd08 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:02.956011 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552dd821-250d-47f2-946b-6624cd91cd08 None admin] PATH_INFO: `/v3/groups/bb43d54bfa084cd59d6ce44ca23cdb96` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:02.957890 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552dd821-250d-47f2-946b-6624cd91cd08 None admin] RBAC: Authorizing `identity:get_group(group_id=bb43d54bfa084cd59d6ce44ca23cdb96)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:02.960206 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552dd821-250d-47f2-946b-6624cd91cd08 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:02.962843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 178/364] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 30 18:15:02 2022] GET /identity/v3/groups/bb43d54bfa084cd59d6ce44ca23cdb96 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:02.963661 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552dd821-250d-47f2-946b-6624cd91cd08 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:02.969175 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 187/365] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:02.970902 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe2847-e624-4313-887b-f6052d554eed None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:03.003316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:03.110034 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552dd821-250d-47f2-946b-6624cd91cd08 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:03.133851 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe2847-e624-4313-887b-f6052d554eed None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:03.167968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51d314-a48a-481b-8207-f4d8dbfb64ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:03.168637 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51d314-a48a-481b-8207-f4d8dbfb64ff None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:03.169173 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51d314-a48a-481b-8207-f4d8dbfb64ff None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:03.169458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51d314-a48a-481b-8207-f4d8dbfb64ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:03.170382 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba51d314-a48a-481b-8207-f4d8dbfb64ff None admin] RBAC: Adding query filter params (role.id=bdbac30ca21b4146bf95b3a053dff30a, user.id=8d64b4d970f04820a2faf68c5d8d349e, scope.system=all) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:03.170696 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba51d314-a48a-481b-8207-f4d8dbfb64ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:03.172153 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba51d314-a48a-481b-8207-f4d8dbfb64ff None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:03.186370 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e09db-1481-4df1-bd0d-1a55a2eefe62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:03.187056 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e09db-1481-4df1-bd0d-1a55a2eefe62 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:03.187331 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 179/366] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Jun 30 18:15:02 2022] GET /identity/v3/role_assignments?user.id=8d64b4d970f04820a2faf68c5d8d349e&scope.system=all&role.id=bdbac30ca21b4146bf95b3a053dff30a => generated 510 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:03.187433 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e09db-1481-4df1-bd0d-1a55a2eefe62 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:03.187756 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e09db-1481-4df1-bd0d-1a55a2eefe62 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:03.188534 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8e09db-1481-4df1-bd0d-1a55a2eefe62 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:03.188766 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51d314-a48a-481b-8207-f4d8dbfb64ff None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:03.188877 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8e09db-1481-4df1-bd0d-1a55a2eefe62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:03.190322 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8e09db-1481-4df1-bd0d-1a55a2eefe62 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:03.207171 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:03.208770 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 188/367] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:03.214044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e09db-1481-4df1-bd0d-1a55a2eefe62 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:03.345304 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51d314-a48a-481b-8207-f4d8dbfb64ff None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:03.354113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:03.370795 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e09db-1481-4df1-bd0d-1a55a2eefe62 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:03.392439 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f243cb-7b57-4300-85c3-e9738fc9927c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:03.394325 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f243cb-7b57-4300-85c3-e9738fc9927c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:03.394829 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f243cb-7b57-4300-85c3-e9738fc9927c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:03.395323 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f243cb-7b57-4300-85c3-e9738fc9927c None admin] PATH_INFO: `/v3/groups/bb43d54bfa084cd59d6ce44ca23cdb96` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:03.397546 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f243cb-7b57-4300-85c3-e9738fc9927c None admin] RBAC: Authorizing `identity:get_group(group_id=bb43d54bfa084cd59d6ce44ca23cdb96)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:03.399329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f243cb-7b57-4300-85c3-e9738fc9927c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:03.401848 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 180/368] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 30 18:15:03 2022] GET /identity/v3/groups/bb43d54bfa084cd59d6ce44ca23cdb96 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:03.402921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f243cb-7b57-4300-85c3-e9738fc9927c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:03.415835 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de7178-145d-4170-84c7-f62ea6aa92b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:03.416202 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de7178-145d-4170-84c7-f62ea6aa92b6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:03.416438 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de7178-145d-4170-84c7-f62ea6aa92b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:03.416725 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de7178-145d-4170-84c7-f62ea6aa92b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:03.419606 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de7178-145d-4170-84c7-f62ea6aa92b6 None admin] RBAC: Adding query filter params (group.id=bb43d54bfa084cd59d6ce44ca23cdb96, role.id=3fbb71ae2de04e78b68d6d6f407ef09c, scope.project.id=d5962edfa0c94d88a1056c1ed5e05058) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:03.419606 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de7178-145d-4170-84c7-f62ea6aa92b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:03.420922 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de7178-145d-4170-84c7-f62ea6aa92b6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:03.429449 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 189/369] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 30 18:15:03 2022] GET /identity/v3/role_assignments?group.id=bb43d54bfa084cd59d6ce44ca23cdb96&scope.project.id=d5962edfa0c94d88a1056c1ed5e05058&role.id=3fbb71ae2de04e78b68d6d6f407ef09c => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:03.430442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de7178-145d-4170-84c7-f62ea6aa92b6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:03.507068 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:03.549531 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f243cb-7b57-4300-85c3-e9738fc9927c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:03.551845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de7178-145d-4170-84c7-f62ea6aa92b6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:03.594956 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8b770-5f7a-42e5-93f5-1d9c16cbad1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:03.595974 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8b770-5f7a-42e5-93f5-1d9c16cbad1e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:03.596289 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8b770-5f7a-42e5-93f5-1d9c16cbad1e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:03.596575 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8b770-5f7a-42e5-93f5-1d9c16cbad1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:03.603450 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2df3b-9169-4145-a7ff-fd312775d87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:03.604410 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2df3b-9169-4145-a7ff-fd312775d87c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:03.604763 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2df3b-9169-4145-a7ff-fd312775d87c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:03.605111 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2df3b-9169-4145-a7ff-fd312775d87c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:03.619662 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2df3b-9169-4145-a7ff-fd312775d87c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:03.627967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4bb2df3b-9169-4145-a7ff-fd312775d87c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:15:03.627967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:03.627967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:03.627967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:03.627967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:03.627967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:03.627967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:03.627967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:03.627967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:03.627967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:03.627967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:03.627967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:03.627967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:03.627967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:03.627967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:03.627967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:03.629222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:03.629222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:03.629222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:03.629222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:03.629222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:03.629222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:03.629222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:03.629222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:03.629222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:03.629222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:03.629222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:03.629222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:03.629222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:03.629222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:03.629222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:03.629222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:03.630714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:03.630714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:03.630714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:03.630714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:03.630714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:03.630714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:03.630714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:03.630714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:03.630714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:03.630714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:03.630714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:15:03.630714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:03.633243 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:03.633485 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 181/370] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:15:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:03.637701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2df3b-9169-4145-a7ff-fd312775d87c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:03.691590 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a8b770-5f7a-42e5-93f5-1d9c16cbad1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:03.693969 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a8b770-5f7a-42e5-93f5-1d9c16cbad1e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:03.770101 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2df3b-9169-4145-a7ff-fd312775d87c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:03.820061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b28109-bb6e-449c-a618-98e2be67ecdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:03.820982 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b28109-bb6e-449c-a618-98e2be67ecdd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:03.821360 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b28109-bb6e-449c-a618-98e2be67ecdd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:03.821729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b28109-bb6e-449c-a618-98e2be67ecdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:03.822692 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b28109-bb6e-449c-a618-98e2be67ecdd None admin] RBAC: Adding query filter params (group.id=bb43d54bfa084cd59d6ce44ca23cdb96, role.id=3fbb71ae2de04e78b68d6d6f407ef09c, scope.project.id=9beede2985d54385bc4532fb19a265a5) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:03.823113 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b28109-bb6e-449c-a618-98e2be67ecdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:03.824731 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b28109-bb6e-449c-a618-98e2be67ecdd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:03.827719 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:03.833495 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 190/371] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:15:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:15:03.835222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8b770-5f7a-42e5-93f5-1d9c16cbad1e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:03.836452 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 182/372] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 30 18:15:03 2022] GET /identity/v3/role_assignments?group.id=bb43d54bfa084cd59d6ce44ca23cdb96&scope.project.id=9beede2985d54385bc4532fb19a265a5&role.id=3fbb71ae2de04e78b68d6d6f407ef09c => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:03.847401 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b28109-bb6e-449c-a618-98e2be67ecdd None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:03.929747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:04.014996 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8b770-5f7a-42e5-93f5-1d9c16cbad1e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:04.055478 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b28109-bb6e-449c-a618-98e2be67ecdd None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:04.077950 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49f296-1b7f-4df5-8fb2-707a0bd4d22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:04.078711 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49f296-1b7f-4df5-8fb2-707a0bd4d22c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:04.078996 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49f296-1b7f-4df5-8fb2-707a0bd4d22c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:04.079282 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49f296-1b7f-4df5-8fb2-707a0bd4d22c None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:04.080054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49f296-1b7f-4df5-8fb2-707a0bd4d22c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:04.080363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49f296-1b7f-4df5-8fb2-707a0bd4d22c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:04.082689 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49f296-1b7f-4df5-8fb2-707a0bd4d22c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:04.105564 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 191/373] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:15:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:04.114677 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49f296-1b7f-4df5-8fb2-707a0bd4d22c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:04.133162 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2483817-d757-439e-9d60-c692ee2e5e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:04.134449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2483817-d757-439e-9d60-c692ee2e5e1c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:04.134920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2483817-d757-439e-9d60-c692ee2e5e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:04.135440 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2483817-d757-439e-9d60-c692ee2e5e1c None admin] PATH_INFO: `/v3/projects/9beede2985d54385bc4532fb19a265a5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:04.138570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2483817-d757-439e-9d60-c692ee2e5e1c None admin] RBAC: Authorizing `identity:get_project(project_id=9beede2985d54385bc4532fb19a265a5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:04.142432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2483817-d757-439e-9d60-c692ee2e5e1c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:04.144830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 183/374] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:15:03 2022] GET /identity/v3/projects/9beede2985d54385bc4532fb19a265a5 => generated 302 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:04.153088 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2483817-d757-439e-9d60-c692ee2e5e1c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:04.345702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:04.366252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49f296-1b7f-4df5-8fb2-707a0bd4d22c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:04.395953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:04.432894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2483817-d757-439e-9d60-c692ee2e5e1c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:04.436150 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ce44d-cd0c-41e5-a78c-87889f401bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:04.436810 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ce44d-cd0c-41e5-a78c-87889f401bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:04.437099 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ce44d-cd0c-41e5-a78c-87889f401bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:04.437384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ce44d-cd0c-41e5-a78c-87889f401bf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:04.513152 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4411a-f7c6-434d-bc91-9d43bc59e89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:04.514091 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4411a-f7c6-434d-bc91-9d43bc59e89e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:04.514391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4411a-f7c6-434d-bc91-9d43bc59e89e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:04.514672 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4411a-f7c6-434d-bc91-9d43bc59e89e None admin] PATH_INFO: `/v3/groups/bb43d54bfa084cd59d6ce44ca23cdb96` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:04.516855 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4411a-f7c6-434d-bc91-9d43bc59e89e None admin] RBAC: Authorizing `identity:get_group(group_id=bb43d54bfa084cd59d6ce44ca23cdb96)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:04.518414 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4411a-f7c6-434d-bc91-9d43bc59e89e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:04.525469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 184/375] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 30 18:15:04 2022] GET /identity/v3/groups/bb43d54bfa084cd59d6ce44ca23cdb96 => generated 226 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:04.530675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4411a-f7c6-434d-bc91-9d43bc59e89e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:04.570865 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ce44d-cd0c-41e5-a78c-87889f401bf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:04.572872 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ce44d-cd0c-41e5-a78c-87889f401bf7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:04.703308 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 192/376] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:15:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:15:04.710609 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ce44d-cd0c-41e5-a78c-87889f401bf7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:04.746749 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:04.832704 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4411a-f7c6-434d-bc91-9d43bc59e89e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:04.893269 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ce44d-cd0c-41e5-a78c-87889f401bf7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:04.912128 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ded32a-0ae1-495a-a664-ae47aa27bd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:04.912909 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ded32a-0ae1-495a-a664-ae47aa27bd4c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:04.913316 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ded32a-0ae1-495a-a664-ae47aa27bd4c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:04.913623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ded32a-0ae1-495a-a664-ae47aa27bd4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:04.914491 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ded32a-0ae1-495a-a664-ae47aa27bd4c None admin] RBAC: Adding query filter params (group.id=bb43d54bfa084cd59d6ce44ca23cdb96, role.id=d5f874a3d78f4ee0ade318e1bbebf7ec, scope.project.id=9beede2985d54385bc4532fb19a265a5) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:04.914924 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ded32a-0ae1-495a-a664-ae47aa27bd4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:04.916543 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ded32a-0ae1-495a-a664-ae47aa27bd4c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:04.919064 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:04.936589 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 185/377] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 30 18:15:04 2022] GET /identity/v3/role_assignments?group.id=bb43d54bfa084cd59d6ce44ca23cdb96&scope.project.id=9beede2985d54385bc4532fb19a265a5&role.id=d5f874a3d78f4ee0ade318e1bbebf7ec => generated 260 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:04.940075 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b305f1-2041-4c2b-a810-23b403667942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:04.940979 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b305f1-2041-4c2b-a810-23b403667942 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:04.941262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b305f1-2041-4c2b-a810-23b403667942 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:04.941603 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b305f1-2041-4c2b-a810-23b403667942 None admin] PATH_INFO: `/v3/projects/6ed5e6c375de4a6186ba026da89db56b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:04.942948 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b305f1-2041-4c2b-a810-23b403667942 None admin] RBAC: Authorizing `identity:get_project(project_id=6ed5e6c375de4a6186ba026da89db56b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:04.945403 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b305f1-2041-4c2b-a810-23b403667942 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:04.947967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 193/378] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:15:04 2022] GET /identity/v3/projects/6ed5e6c375de4a6186ba026da89db56b => generated 344 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:04.956311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ded32a-0ae1-495a-a664-ae47aa27bd4c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:05.130723 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ded32a-0ae1-495a-a664-ae47aa27bd4c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:05.174382 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a33432-9982-45c4-8449-5c3956c30254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:05.175266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a33432-9982-45c4-8449-5c3956c30254 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:05.175554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a33432-9982-45c4-8449-5c3956c30254 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:05.175837 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a33432-9982-45c4-8449-5c3956c30254 None admin] PATH_INFO: `/v3/groups/05a2807df9df46cd92b4efb85984b78c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:05.186252 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a33432-9982-45c4-8449-5c3956c30254 None admin] RBAC: Authorizing `identity:get_group(group_id=05a2807df9df46cd92b4efb85984b78c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:05.187725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a33432-9982-45c4-8449-5c3956c30254 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:05.189826 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 186/379] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 30 18:15:04 2022] GET /identity/v3/groups/05a2807df9df46cd92b4efb85984b78c => generated 229 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:05.196102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b305f1-2041-4c2b-a810-23b403667942 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:05.325690 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:05.348201 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b305f1-2041-4c2b-a810-23b403667942 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:05.371244 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5cff5e-13da-4b0a-bd9c-e2accf92713c None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:05.371597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5cff5e-13da-4b0a-bd9c-e2accf92713c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:05.371807 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5cff5e-13da-4b0a-bd9c-e2accf92713c None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:05.373621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 187/380] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:05.389613 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93352a0a-a2e1-4291-a3a9-d68ef06951b8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:05.389613 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93352a0a-a2e1-4291-a3a9-d68ef06951b8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:05.389613 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93352a0a-a2e1-4291-a3a9-d68ef06951b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:05.389951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f2ca3-791c-47a1-b5ca-e5f735a5a91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:05.390215 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f2ca3-791c-47a1-b5ca-e5f735a5a91b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:05.390510 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f2ca3-791c-47a1-b5ca-e5f735a5a91b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:05.390798 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f2ca3-791c-47a1-b5ca-e5f735a5a91b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:05.391655 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f2ca3-791c-47a1-b5ca-e5f735a5a91b None admin] RBAC: Adding query filter params (group.id=05a2807df9df46cd92b4efb85984b78c, role.id=0ef3687af20a443281e3cfdceca3597d, scope.project.id=6ed5e6c375de4a6186ba026da89db56b) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:05.391974 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f2ca3-791c-47a1-b5ca-e5f735a5a91b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:05.393452 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f2ca3-791c-47a1-b5ca-e5f735a5a91b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:05.402662 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 194/381] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 30 18:15:05 2022] GET /identity/v3/role_assignments?group.id=05a2807df9df46cd92b4efb85984b78c&scope.project.id=6ed5e6c375de4a6186ba026da89db56b&role.id=0ef3687af20a443281e3cfdceca3597d => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:05.415985 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-93352a0a-a2e1-4291-a3a9-d68ef06951b8 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:05.417383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-93352a0a-a2e1-4291-a3a9-d68ef06951b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:05.563607 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:05.594002 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 188/382] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:05.628866 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f2ca3-791c-47a1-b5ca-e5f735a5a91b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:05.783690 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f2ca3-791c-47a1-b5ca-e5f735a5a91b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:05.839457 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ddf002-3a8d-4f34-b9e9-9e73d24abf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:05.840547 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddf002-3a8d-4f34-b9e9-9e73d24abf42 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:05.840985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddf002-3a8d-4f34-b9e9-9e73d24abf42 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:05.841416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddf002-3a8d-4f34-b9e9-9e73d24abf42 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:05.867404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ddf002-3a8d-4f34-b9e9-9e73d24abf42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:05.876054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f5ddf002-3a8d-4f34-b9e9-9e73d24abf42 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:15:05.876054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:05.876054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:05.876054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:05.876054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:05.876054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:05.876054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:05.876054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:05.876054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:05.876054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:05.876054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:05.876054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:05.876054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:05.876054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:05.876054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:05.876054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:05.877176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:05.877176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:05.877176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:05.877176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:05.877176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:05.877176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:05.877176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:05.877176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:05.877176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:05.877176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:05.877176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:05.877176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:05.877176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:05.877176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:05.877176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:05.877176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:05.878212 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:05.878212 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:05.878212 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:05.878212 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:05.878212 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:05.878212 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:05.878212 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:05.878212 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:05.878212 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:05.878212 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:05.878212 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:15:05.878212 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:05.880704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 195/383] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 30 18:15:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:05.891478 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93352a0a-a2e1-4291-a3a9-d68ef06951b8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:06.106242 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93352a0a-a2e1-4291-a3a9-d68ef06951b8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:06.161528 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a2f19-5bf5-40fe-bde3-629c3ba1de7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:06.163321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a2f19-5bf5-40fe-bde3-629c3ba1de7e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:06.163820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a2f19-5bf5-40fe-bde3-629c3ba1de7e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:06.164301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a2f19-5bf5-40fe-bde3-629c3ba1de7e None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:06.165308 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a2f19-5bf5-40fe-bde3-629c3ba1de7e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:06.165857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a2f19-5bf5-40fe-bde3-629c3ba1de7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:06.167481 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a2f19-5bf5-40fe-bde3-629c3ba1de7e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:06.184508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 189/384] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:06.189784 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ddf002-3a8d-4f34-b9e9-9e73d24abf42 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:06.220237 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:06.380120 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d36ad-2502-4ab9-80bb-c47b6b824741 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:06.380662 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d36ad-2502-4ab9-80bb-c47b6b824741 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:06.381098 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d36ad-2502-4ab9-80bb-c47b6b824741 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:06.383284 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 190/385] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:06.385884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ddf002-3a8d-4f34-b9e9-9e73d24abf42 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:06.389321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bc036-dec5-4ba2-8744-f87cec8ee0ef None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:06.389996 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bc036-dec5-4ba2-8744-f87cec8ee0ef None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:06.390475 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bc036-dec5-4ba2-8744-f87cec8ee0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:06.413732 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e35bc036-dec5-4ba2-8744-f87cec8ee0ef None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:06.416602 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e35bc036-dec5-4ba2-8744-f87cec8ee0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:06.467548 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd84f4-59b8-40c9-b3b6-7f1468e757b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:06.468915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd84f4-59b8-40c9-b3b6-7f1468e757b4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:06.470607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd84f4-59b8-40c9-b3b6-7f1468e757b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:06.471304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd84f4-59b8-40c9-b3b6-7f1468e757b4 None admin] PATH_INFO: `/v3/groups/bb43d54bfa084cd59d6ce44ca23cdb96` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:06.474032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd84f4-59b8-40c9-b3b6-7f1468e757b4 None admin] RBAC: Authorizing `identity:get_group(group_id=bb43d54bfa084cd59d6ce44ca23cdb96)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:06.476302 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd84f4-59b8-40c9-b3b6-7f1468e757b4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:06.479397 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 196/386] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 30 18:15:06 2022] GET /identity/v3/groups/bb43d54bfa084cd59d6ce44ca23cdb96 => generated 226 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:06.488462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd84f4-59b8-40c9-b3b6-7f1468e757b4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:06.544014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:06.603471 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 191/387] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:06.606527 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:06.642403 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd84f4-59b8-40c9-b3b6-7f1468e757b4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:06.650700 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35bc036-dec5-4ba2-8744-f87cec8ee0ef None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:06.735255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39086c71-8ecc-46cb-8880-3dda87ba63a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:06.737194 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39086c71-8ecc-46cb-8880-3dda87ba63a4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:06.738966 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39086c71-8ecc-46cb-8880-3dda87ba63a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:06.740681 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39086c71-8ecc-46cb-8880-3dda87ba63a4 None admin] PATH_INFO: `/v3/projects/d5962edfa0c94d88a1056c1ed5e05058` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:06.744914 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39086c71-8ecc-46cb-8880-3dda87ba63a4 None admin] RBAC: Authorizing `identity:get_project(project_id=d5962edfa0c94d88a1056c1ed5e05058)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:06.749702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39086c71-8ecc-46cb-8880-3dda87ba63a4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:06.755970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 197/388] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:15:06 2022] GET /identity/v3/projects/d5962edfa0c94d88a1056c1ed5e05058 => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:06.758975 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c41ce-61bd-47f7-8847-f82af44fb6e5 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:06.759298 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c41ce-61bd-47f7-8847-f82af44fb6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:06.759546 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c41ce-61bd-47f7-8847-f82af44fb6e5 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:06.761206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 198/389] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:06.765707 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c41ce-61bd-47f7-8847-f82af44fb6e5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:06.843246 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35bc036-dec5-4ba2-8744-f87cec8ee0ef None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:06.927482 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf22d5d-8968-41fe-8c18-7cdbca923c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:06.928553 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf22d5d-8968-41fe-8c18-7cdbca923c71 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:06.929460 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf22d5d-8968-41fe-8c18-7cdbca923c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:06.929714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf22d5d-8968-41fe-8c18-7cdbca923c71 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:06.950187 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf22d5d-8968-41fe-8c18-7cdbca923c71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:06.960048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-dbf22d5d-8968-41fe-8c18-7cdbca923c71 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:15:06.960048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:06.960048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:06.960048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:06.960048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:06.960048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:06.960048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:06.960048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:06.960048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:06.960048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:06.960048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:06.960048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:06.960048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:06.960048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:06.960048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:06.960048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:06.961847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:06.961847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:06.961847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:06.961847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:06.961847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:06.961847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:06.961847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:06.961847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:06.961847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:06.961847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:06.961847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:06.961847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:06.961847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:06.961847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:06.961847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:06.961847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:06.963450 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:06.963450 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:06.963450 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:06.963450 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:06.963450 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:06.963450 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:06.963450 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:06.963450 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:06.963450 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:06.963450 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:06.963450 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:15:06.963450 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:06.963450 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 192/390] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:15:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:06.964771 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712b962-c8b0-4ef2-8e79-3dbbe51d7d73 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:06.964771 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712b962-c8b0-4ef2-8e79-3dbbe51d7d73 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:06.964771 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712b962-c8b0-4ef2-8e79-3dbbe51d7d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:06.979325 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c41ce-61bd-47f7-8847-f82af44fb6e5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:06.986666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3712b962-c8b0-4ef2-8e79-3dbbe51d7d73 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:06.987818 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3712b962-c8b0-4ef2-8e79-3dbbe51d7d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:07.042960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:07.056028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a162d2e-357b-42e6-ad78-667225a068cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:07.056958 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a162d2e-357b-42e6-ad78-667225a068cc None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:07.057238 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a162d2e-357b-42e6-ad78-667225a068cc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:07.057524 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a162d2e-357b-42e6-ad78-667225a068cc None admin] PATH_INFO: `/v3/projects/d5962edfa0c94d88a1056c1ed5e05058/groups/bb43d54bfa084cd59d6ce44ca23cdb96/roles/d5f874a3d78f4ee0ade318e1bbebf7ec` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:07.060327 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a162d2e-357b-42e6-ad78-667225a068cc None admin] RBAC: Authorizing `identity:create_grant(project_id=d5962edfa0c94d88a1056c1ed5e05058, group_id=bb43d54bfa084cd59d6ce44ca23cdb96, role_id=d5f874a3d78f4ee0ade318e1bbebf7ec)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:07.062510 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a162d2e-357b-42e6-ad78-667225a068cc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:07.079238 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 199/391] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Jun 30 18:15:06 2022] PUT /identity/v3/projects/d5962edfa0c94d88a1056c1ed5e05058/groups/bb43d54bfa084cd59d6ce44ca23cdb96/roles/d5f874a3d78f4ee0ade318e1bbebf7ec => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:15:07.080210 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a162d2e-357b-42e6-ad78-667225a068cc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:07.085146 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:07.135280 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 193/392] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:07.220874 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712b962-c8b0-4ef2-8e79-3dbbe51d7d73 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:07.244947 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a162d2e-357b-42e6-ad78-667225a068cc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:07.295078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab808e68-8e43-434e-b6d7-f47289f45c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:07.295756 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab808e68-8e43-434e-b6d7-f47289f45c30 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:07.296119 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab808e68-8e43-434e-b6d7-f47289f45c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:07.296391 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab808e68-8e43-434e-b6d7-f47289f45c30 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:07.297117 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab808e68-8e43-434e-b6d7-f47289f45c30 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:07.298396 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab808e68-8e43-434e-b6d7-f47289f45c30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:07.299049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab808e68-8e43-434e-b6d7-f47289f45c30 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:07.310016 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 200/393] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:15:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:07.326591 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab808e68-8e43-434e-b6d7-f47289f45c30 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:07.456180 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:07.477076 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712b962-c8b0-4ef2-8e79-3dbbe51d7d73 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:07.508668 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab808e68-8e43-434e-b6d7-f47289f45c30 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:07.517299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7dfd02-9932-45a7-a89a-4c1531f7c375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:07.517795 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7dfd02-9932-45a7-a89a-4c1531f7c375 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:07.518079 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7dfd02-9932-45a7-a89a-4c1531f7c375 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:07.518355 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7dfd02-9932-45a7-a89a-4c1531f7c375 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:07.535121 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7dfd02-9932-45a7-a89a-4c1531f7c375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:07.552223 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a04d0-6607-44be-ade3-a0a6c5bd6ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:07.554789 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a04d0-6607-44be-ade3-a0a6c5bd6ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:07.555300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a04d0-6607-44be-ade3-a0a6c5bd6ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:07.555803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a04d0-6607-44be-ade3-a0a6c5bd6ab5 None admin] PATH_INFO: `/v3/groups/bb43d54bfa084cd59d6ce44ca23cdb96` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:07.556266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-be7dfd02-9932-45a7-a89a-4c1531f7c375 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:15:07.556266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:07.556266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:07.556266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:07.556266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:07.556266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:07.556266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:07.556266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:07.556266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:07.556266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:07.556266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:07.556266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:07.556266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:07.556266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:07.556266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:07.556266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:07.557326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:07.557326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:07.557326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:07.557326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:07.557326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:07.557326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:07.557326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:07.557326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:07.557326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:07.557326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:07.557326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:07.557326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:07.557326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:07.557326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:07.557326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:07.557326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:07.558271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:07.558271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:07.558271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:07.558271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:07.558271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:07.558271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:07.558271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:07.558271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:07.558271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:07.558271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:07.558271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:15:07.558271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:07.558271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 194/394] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:15:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:07.560793 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774a04d0-6607-44be-ade3-a0a6c5bd6ab5 None admin] RBAC: Authorizing `identity:get_group(group_id=bb43d54bfa084cd59d6ce44ca23cdb96)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:07.569639 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774a04d0-6607-44be-ade3-a0a6c5bd6ab5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:07.569839 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 201/395] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 30 18:15:07 2022] GET /identity/v3/groups/bb43d54bfa084cd59d6ce44ca23cdb96 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:15:07.571025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7dfd02-9932-45a7-a89a-4c1531f7c375 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:07.571427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a04d0-6607-44be-ade3-a0a6c5bd6ab5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:07.661523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:07.796316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a04d0-6607-44be-ade3-a0a6c5bd6ab5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:07.844990 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ac356-025e-469f-a064-63a677b2bb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:07.845902 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ac356-025e-469f-a064-63a677b2bb05 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:07.846189 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ac356-025e-469f-a064-63a677b2bb05 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:07.846764 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ac356-025e-469f-a064-63a677b2bb05 None admin] PATH_INFO: `/v3/projects/d5962edfa0c94d88a1056c1ed5e05058` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:07.848041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ac356-025e-469f-a064-63a677b2bb05 None admin] RBAC: Authorizing `identity:get_project(project_id=d5962edfa0c94d88a1056c1ed5e05058)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:07.848155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7dfd02-9932-45a7-a89a-4c1531f7c375 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:07.849520 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ac356-025e-469f-a064-63a677b2bb05 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:07.857398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 202/396] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:15:07 2022] GET /identity/v3/projects/d5962edfa0c94d88a1056c1ed5e05058 => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:15:07.858725 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ac356-025e-469f-a064-63a677b2bb05 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:07.861570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:07.931448 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab4070-0b6f-4870-baa5-b1600172dd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:07.932312 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab4070-0b6f-4870-baa5-b1600172dd80 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:07.933024 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab4070-0b6f-4870-baa5-b1600172dd80 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:07.933309 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab4070-0b6f-4870-baa5-b1600172dd80 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:07.934413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdab4070-0b6f-4870-baa5-b1600172dd80 None admin] RBAC: Adding query filter params (name=member) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:07.934726 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdab4070-0b6f-4870-baa5-b1600172dd80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:07.937028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdab4070-0b6f-4870-baa5-b1600172dd80 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:07.948651 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 195/397] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:15:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:07.950118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80900561-df2d-48e1-9553-89b3243c9c77 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:07.950407 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80900561-df2d-48e1-9553-89b3243c9c77 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:07.950688 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80900561-df2d-48e1-9553-89b3243c9c77 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:07.957742 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 196/398] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:07.958760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80900561-df2d-48e1-9553-89b3243c9c77 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:07.996095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ac356-025e-469f-a064-63a677b2bb05 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:08.054315 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef9b9b-7cfd-42cd-bd46-ee90b9380ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:08.055417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef9b9b-7cfd-42cd-bd46-ee90b9380ffb None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:08.055715 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef9b9b-7cfd-42cd-bd46-ee90b9380ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:08.055987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef9b9b-7cfd-42cd-bd46-ee90b9380ffb None admin] PATH_INFO: `/v3/projects/d5962edfa0c94d88a1056c1ed5e05058/groups/bb43d54bfa084cd59d6ce44ca23cdb96/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:08.058144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef9b9b-7cfd-42cd-bd46-ee90b9380ffb None admin] RBAC: Authorizing `identity:create_grant(project_id=d5962edfa0c94d88a1056c1ed5e05058, group_id=bb43d54bfa084cd59d6ce44ca23cdb96, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:08.060008 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef9b9b-7cfd-42cd-bd46-ee90b9380ffb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:08.088913 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 203/399] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Jun 30 18:15:07 2022] PUT /identity/v3/projects/d5962edfa0c94d88a1056c1ed5e05058/groups/bb43d54bfa084cd59d6ce44ca23cdb96/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:15:08.088913 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955220c-c1a4-4ce6-835c-f074e9cfc973 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:08.088913 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955220c-c1a4-4ce6-835c-f074e9cfc973 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:08.088913 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955220c-c1a4-4ce6-835c-f074e9cfc973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:08.113337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:08.132691 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80900561-df2d-48e1-9553-89b3243c9c77 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:08.135122 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9955220c-c1a4-4ce6-835c-f074e9cfc973 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:08.138200 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9955220c-c1a4-4ce6-835c-f074e9cfc973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:08.218562 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557096da-4c7e-456b-9172-63b765a7f63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:08.219869 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557096da-4c7e-456b-9172-63b765a7f63b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:08.220748 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557096da-4c7e-456b-9172-63b765a7f63b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:08.221237 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557096da-4c7e-456b-9172-63b765a7f63b None admin] PATH_INFO: `/v3/groups/bb43d54bfa084cd59d6ce44ca23cdb96` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:08.223473 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557096da-4c7e-456b-9172-63b765a7f63b None admin] RBAC: Authorizing `identity:get_group(group_id=bb43d54bfa084cd59d6ce44ca23cdb96)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:08.226164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557096da-4c7e-456b-9172-63b765a7f63b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:08.228624 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 197/400] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 30 18:15:07 2022] GET /identity/v3/groups/bb43d54bfa084cd59d6ce44ca23cdb96 => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:08.230331 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34e591-2ccb-4aae-b54e-0b0a8e5723bc None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:08.230818 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34e591-2ccb-4aae-b54e-0b0a8e5723bc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:08.231291 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34e591-2ccb-4aae-b54e-0b0a8e5723bc None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:08.233292 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 198/401] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:08.236839 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34e591-2ccb-4aae-b54e-0b0a8e5723bc None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:08.271721 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:08.291608 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 204/402] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:08.293978 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa3d66-dec2-45d5-9fa4-110bd4751517 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:08.294283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa3d66-dec2-45d5-9fa4-110bd4751517 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:08.294569 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa3d66-dec2-45d5-9fa4-110bd4751517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:08.314077 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-72fa3d66-dec2-45d5-9fa4-110bd4751517 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:08.315241 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-72fa3d66-dec2-45d5-9fa4-110bd4751517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:08.408133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34e591-2ccb-4aae-b54e-0b0a8e5723bc None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:08.420942 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 205/403] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:08.422820 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa3d66-dec2-45d5-9fa4-110bd4751517 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:08.455885 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611eea9d-91ca-4102-ab67-e97a969c3e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:08.457480 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611eea9d-91ca-4102-ab67-e97a969c3e8f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:08.458031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611eea9d-91ca-4102-ab67-e97a969c3e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:08.458515 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611eea9d-91ca-4102-ab67-e97a969c3e8f None admin] PATH_INFO: `/v3/projects/9beede2985d54385bc4532fb19a265a5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:08.460627 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611eea9d-91ca-4102-ab67-e97a969c3e8f None admin] RBAC: Authorizing `identity:get_project(project_id=9beede2985d54385bc4532fb19a265a5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:08.463002 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611eea9d-91ca-4102-ab67-e97a969c3e8f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:08.465472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 199/404] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:15:08 2022] GET /identity/v3/projects/9beede2985d54385bc4532fb19a265a5 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:08.466666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611eea9d-91ca-4102-ab67-e97a969c3e8f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:08.525970 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:08.611941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611eea9d-91ca-4102-ab67-e97a969c3e8f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:08.623004 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa3d66-dec2-45d5-9fa4-110bd4751517 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:08.658926 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6647e2-0f49-4724-a2aa-72b06459f7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:08.660655 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6647e2-0f49-4724-a2aa-72b06459f7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:08.661071 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6647e2-0f49-4724-a2aa-72b06459f7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:08.661491 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6647e2-0f49-4724-a2aa-72b06459f7b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:08.665649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:08.677614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6647e2-0f49-4724-a2aa-72b06459f7b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:08.687705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-eb6647e2-0f49-4724-a2aa-72b06459f7b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:15:08.687705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:08.687705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:08.687705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:08.687705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:08.687705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:08.687705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:08.687705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:08.687705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:08.687705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:08.687705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:08.687705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:08.687705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:08.687705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:08.687705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:08.687705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:08.689872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:08.689872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:08.689872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:08.689872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:08.689872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:08.689872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:08.689872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:08.689872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:08.689872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:08.689872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:08.689872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:08.689872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:08.689872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:08.689872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:08.689872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:08.689872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:08.691444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:08.691444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:08.691444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:08.691444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:08.691444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:08.691444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:08.691444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:08.691444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:08.691444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:08.691444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:08.691444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:15:08.691444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:08.691444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 200/405] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:15:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:08.693996 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6647e2-0f49-4724-a2aa-72b06459f7b9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:08.698932 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d8606-e286-4a26-989e-1e081a1bb464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:08.700869 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d8606-e286-4a26-989e-1e081a1bb464 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:08.701775 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d8606-e286-4a26-989e-1e081a1bb464 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:08.702561 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d8606-e286-4a26-989e-1e081a1bb464 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:08.727232 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d8606-e286-4a26-989e-1e081a1bb464 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:08.743684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-143d8606-e286-4a26-989e-1e081a1bb464 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:15:08.743684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:08.743684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:08.743684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:08.743684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:08.743684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:08.743684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:08.743684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:08.743684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:08.743684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:08.743684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:08.743684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:08.743684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:08.743684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:08.743684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:08.743684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:08.744632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:08.744632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:08.744632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:08.744632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:08.744632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:08.744632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:08.744632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:08.744632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:08.744632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:08.744632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:08.744632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:08.744632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:08.744632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:08.744632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:08.744632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:08.744632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:08.745543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:08.745543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:08.745543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:08.745543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:08.745543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:08.745543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:08.745543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:08.745543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:08.745543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:08.745543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:08.745543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:15:08.745543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:08.753922 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 206/406] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 30 18:15:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:08.755207 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d8606-e286-4a26-989e-1e081a1bb464 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:08.856017 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:08.871339 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6647e2-0f49-4724-a2aa-72b06459f7b9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:08.918392 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58cb86-aea6-4397-a510-047b8693ea3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:08.919934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58cb86-aea6-4397-a510-047b8693ea3c None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:08.920623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58cb86-aea6-4397-a510-047b8693ea3c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:08.921094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58cb86-aea6-4397-a510-047b8693ea3c None admin] PATH_INFO: `/v3/projects/9beede2985d54385bc4532fb19a265a5/groups/bb43d54bfa084cd59d6ce44ca23cdb96/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:08.924218 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58cb86-aea6-4397-a510-047b8693ea3c None admin] RBAC: Authorizing `identity:create_grant(project_id=9beede2985d54385bc4532fb19a265a5, group_id=bb43d54bfa084cd59d6ce44ca23cdb96, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:08.925986 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58cb86-aea6-4397-a510-047b8693ea3c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:08.927559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d8606-e286-4a26-989e-1e081a1bb464 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:08.954433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 201/407] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Jun 30 18:15:08 2022] PUT /identity/v3/projects/9beede2985d54385bc4532fb19a265a5/groups/bb43d54bfa084cd59d6ce44ca23cdb96/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:15:08.956458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58cb86-aea6-4397-a510-047b8693ea3c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:09.051285 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362f1bb-cb6f-4fad-b371-7465f9ca7648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:09.052262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362f1bb-cb6f-4fad-b371-7465f9ca7648 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:09.052736 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362f1bb-cb6f-4fad-b371-7465f9ca7648 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:09.053342 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362f1bb-cb6f-4fad-b371-7465f9ca7648 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:09.054584 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7362f1bb-cb6f-4fad-b371-7465f9ca7648 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:09.055046 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7362f1bb-cb6f-4fad-b371-7465f9ca7648 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:09.060141 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7362f1bb-cb6f-4fad-b371-7465f9ca7648 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:09.105223 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 207/408] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:15:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:09.112263 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362f1bb-cb6f-4fad-b371-7465f9ca7648 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:09.212318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:09.285614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362f1bb-cb6f-4fad-b371-7465f9ca7648 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:09.287331 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58cb86-aea6-4397-a510-047b8693ea3c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:09.388341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab37be4-947c-4c65-91bf-669b5b70a9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:09.389532 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab37be4-947c-4c65-91bf-669b5b70a9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:09.390126 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab37be4-947c-4c65-91bf-669b5b70a9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:09.390637 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab37be4-947c-4c65-91bf-669b5b70a9a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:09.392138 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab37be4-947c-4c65-91bf-669b5b70a9a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:09.392687 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab37be4-947c-4c65-91bf-669b5b70a9a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:09.395453 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab37be4-947c-4c65-91bf-669b5b70a9a3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:09.410110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef463eda-8d71-48ef-98dd-f7317af1782e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:09.411075 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef463eda-8d71-48ef-98dd-f7317af1782e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:09.411416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef463eda-8d71-48ef-98dd-f7317af1782e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:09.411783 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef463eda-8d71-48ef-98dd-f7317af1782e None admin] PATH_INFO: `/v3/groups/05a2807df9df46cd92b4efb85984b78c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:09.414141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 202/409] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 459 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:09.415166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef463eda-8d71-48ef-98dd-f7317af1782e None admin] RBAC: Authorizing `identity:get_group(group_id=05a2807df9df46cd92b4efb85984b78c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:09.416727 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef463eda-8d71-48ef-98dd-f7317af1782e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:09.418613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 208/410] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 30 18:15:09 2022] GET /identity/v3/groups/05a2807df9df46cd92b4efb85984b78c => generated 229 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:09.423619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab37be4-947c-4c65-91bf-669b5b70a9a3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:09.425521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef463eda-8d71-48ef-98dd-f7317af1782e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:09.464727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:09.600303 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef463eda-8d71-48ef-98dd-f7317af1782e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:09.616516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:09.648661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e51b68-93bb-46a9-91eb-d41798aaec3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:09.649674 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e51b68-93bb-46a9-91eb-d41798aaec3b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:09.650151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e51b68-93bb-46a9-91eb-d41798aaec3b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:09.650506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e51b68-93bb-46a9-91eb-d41798aaec3b None admin] PATH_INFO: `/v3/projects/6ed5e6c375de4a6186ba026da89db56b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:09.651916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e51b68-93bb-46a9-91eb-d41798aaec3b None admin] RBAC: Authorizing `identity:get_project(project_id=6ed5e6c375de4a6186ba026da89db56b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:09.653503 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e51b68-93bb-46a9-91eb-d41798aaec3b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:09.655915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 209/411] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:15:09 2022] GET /identity/v3/projects/6ed5e6c375de4a6186ba026da89db56b => generated 344 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:09.661867 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e51b68-93bb-46a9-91eb-d41798aaec3b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:09.764571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab37be4-947c-4c65-91bf-669b5b70a9a3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:09.830172 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16172d76-a722-4948-9909-ce600a847982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:09.831805 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16172d76-a722-4948-9909-ce600a847982 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:09.832289 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16172d76-a722-4948-9909-ce600a847982 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:09.832780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16172d76-a722-4948-9909-ce600a847982 None admin] PATH_INFO: `/v3/groups/bb43d54bfa084cd59d6ce44ca23cdb96` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:09.835532 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16172d76-a722-4948-9909-ce600a847982 None admin] RBAC: Authorizing `identity:get_group(group_id=bb43d54bfa084cd59d6ce44ca23cdb96)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:09.838303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16172d76-a722-4948-9909-ce600a847982 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:09.841512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 203/412] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 30 18:15:09 2022] GET /identity/v3/groups/bb43d54bfa084cd59d6ce44ca23cdb96 => generated 226 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:09.848352 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16172d76-a722-4948-9909-ce600a847982 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:09.983296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e51b68-93bb-46a9-91eb-d41798aaec3b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:10.088425 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b85bb9-d350-43f8-8d9e-d4986365bb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:10.089967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:10.090653 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b85bb9-d350-43f8-8d9e-d4986365bb5c None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:10.090653 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b85bb9-d350-43f8-8d9e-d4986365bb5c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:10.090887 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b85bb9-d350-43f8-8d9e-d4986365bb5c None admin] PATH_INFO: `/v3/projects/6ed5e6c375de4a6186ba026da89db56b/groups/05a2807df9df46cd92b4efb85984b78c/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:10.100014 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b85bb9-d350-43f8-8d9e-d4986365bb5c None admin] RBAC: Authorizing `identity:create_grant(project_id=6ed5e6c375de4a6186ba026da89db56b, group_id=05a2807df9df46cd92b4efb85984b78c, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:10.111275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b85bb9-d350-43f8-8d9e-d4986365bb5c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:10.145282 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 210/413] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Jun 30 18:15:09 2022] PUT /identity/v3/projects/6ed5e6c375de4a6186ba026da89db56b/groups/05a2807df9df46cd92b4efb85984b78c/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:15:10.160091 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16172d76-a722-4948-9909-ce600a847982 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:10.240418 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ac9b7-92fa-4101-b2d5-ccee515e3462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:10.241914 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ac9b7-92fa-4101-b2d5-ccee515e3462 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:10.242220 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ac9b7-92fa-4101-b2d5-ccee515e3462 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:10.242546 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ac9b7-92fa-4101-b2d5-ccee515e3462 None admin] PATH_INFO: `/v3/projects/9beede2985d54385bc4532fb19a265a5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:10.244171 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388ac9b7-92fa-4101-b2d5-ccee515e3462 None admin] RBAC: Authorizing `identity:get_project(project_id=9beede2985d54385bc4532fb19a265a5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:10.245747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388ac9b7-92fa-4101-b2d5-ccee515e3462 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:10.248064 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 204/414] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:15:09 2022] GET /identity/v3/projects/9beede2985d54385bc4532fb19a265a5 => generated 302 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:10.258819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b85bb9-d350-43f8-8d9e-d4986365bb5c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:10.289256 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d7227-75e6-4351-83b7-f7c171e3e85e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:10.289505 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d7227-75e6-4351-83b7-f7c171e3e85e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:10.289823 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d7227-75e6-4351-83b7-f7c171e3e85e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:10.291584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 205/415] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:10.297988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085cee0-5566-4b6a-89a1-3de03c6bbd02 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:10.298272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085cee0-5566-4b6a-89a1-3de03c6bbd02 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:10.298596 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085cee0-5566-4b6a-89a1-3de03c6bbd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:10.331310 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2085cee0-5566-4b6a-89a1-3de03c6bbd02 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:10.333328 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2085cee0-5566-4b6a-89a1-3de03c6bbd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:10.427852 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:10.517726 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 206/416] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 18:15:10.529150 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b85bb9-d350-43f8-8d9e-d4986365bb5c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:10.542538 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085cee0-5566-4b6a-89a1-3de03c6bbd02 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:10.658358 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f613e5-744a-4f78-8f4a-b5d1656c1596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:10.662030 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f613e5-744a-4f78-8f4a-b5d1656c1596 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:10.672072 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f613e5-744a-4f78-8f4a-b5d1656c1596 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:10.673028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f613e5-744a-4f78-8f4a-b5d1656c1596 None admin] PATH_INFO: `/v3/projects/9beede2985d54385bc4532fb19a265a5/groups/bb43d54bfa084cd59d6ce44ca23cdb96/roles/d5f874a3d78f4ee0ade318e1bbebf7ec` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:10.677542 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f613e5-744a-4f78-8f4a-b5d1656c1596 None admin] RBAC: Authorizing `identity:create_grant(project_id=9beede2985d54385bc4532fb19a265a5, group_id=bb43d54bfa084cd59d6ce44ca23cdb96, role_id=d5f874a3d78f4ee0ade318e1bbebf7ec)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:10.681440 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f613e5-744a-4f78-8f4a-b5d1656c1596 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:10.713235 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 211/417] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Jun 30 18:15:10 2022] PUT /identity/v3/projects/9beede2985d54385bc4532fb19a265a5/groups/bb43d54bfa084cd59d6ce44ca23cdb96/roles/d5f874a3d78f4ee0ade318e1bbebf7ec => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:15:10.719880 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862a32c-6b7c-4206-824f-62bbdd7a8a39 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:10.720558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862a32c-6b7c-4206-824f-62bbdd7a8a39 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:10.721165 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862a32c-6b7c-4206-824f-62bbdd7a8a39 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:10.723444 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 212/418] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:10.743532 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876d5dc-e4bb-4e40-9a00-9848904217df None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:10.744253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876d5dc-e4bb-4e40-9a00-9848904217df None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:10.744807 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876d5dc-e4bb-4e40-9a00-9848904217df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:10.775165 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085cee0-5566-4b6a-89a1-3de03c6bbd02 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:10.777956 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:10.796150 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8876d5dc-e4bb-4e40-9a00-9848904217df None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:10.798415 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8876d5dc-e4bb-4e40-9a00-9848904217df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:10.816770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ff4ba-8ba4-4402-8ce9-1893b2d9e57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:10.817677 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ff4ba-8ba4-4402-8ce9-1893b2d9e57d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:10.817967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ff4ba-8ba4-4402-8ce9-1893b2d9e57d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:10.818249 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ff4ba-8ba4-4402-8ce9-1893b2d9e57d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:10.840395 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ff4ba-8ba4-4402-8ce9-1893b2d9e57d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:10.861257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-081ff4ba-8ba4-4402-8ce9-1893b2d9e57d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:15:10.861257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:10.861257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:10.861257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:10.861257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:10.861257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:10.861257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:10.861257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:10.861257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:10.861257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:10.861257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:10.861257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:10.861257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:10.861257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:10.861257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:10.861257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:10.863090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:10.863090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:10.863090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:10.863090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:10.863090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:10.863090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:10.863090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:10.863090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:10.863090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:10.863090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:10.863090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:10.863090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:10.863090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:10.863090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:10.863090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:10.863090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:10.864669 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:10.864669 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:10.864669 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:10.864669 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:10.864669 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:10.864669 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:10.864669 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:10.864669 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:10.864669 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:10.864669 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:10.864669 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:15:10.864669 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:10.864669 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 207/419] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 30 18:15:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:10.873745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ff4ba-8ba4-4402-8ce9-1893b2d9e57d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:11.030153 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 213/420] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:11.065397 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876d5dc-e4bb-4e40-9a00-9848904217df None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:11.108633 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ff4ba-8ba4-4402-8ce9-1893b2d9e57d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:11.118874 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:11.165485 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103dfce-7575-4ed9-8545-d891364848cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:11.166220 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103dfce-7575-4ed9-8545-d891364848cd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:11.166582 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103dfce-7575-4ed9-8545-d891364848cd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:11.166866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103dfce-7575-4ed9-8545-d891364848cd None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:11.167636 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103dfce-7575-4ed9-8545-d891364848cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:11.167950 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103dfce-7575-4ed9-8545-d891364848cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:11.170165 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103dfce-7575-4ed9-8545-d891364848cd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:11.182381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 208/421] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:11.189658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103dfce-7575-4ed9-8545-d891364848cd None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:11.195098 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:11.254563 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876d5dc-e4bb-4e40-9a00-9848904217df None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:11.326176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843df7f3-232f-4bbb-a48e-a178bea06be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:11.327371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843df7f3-232f-4bbb-a48e-a178bea06be4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:11.327943 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843df7f3-232f-4bbb-a48e-a178bea06be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:11.328437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843df7f3-232f-4bbb-a48e-a178bea06be4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:11.352418 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843df7f3-232f-4bbb-a48e-a178bea06be4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:11.366717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-843df7f3-232f-4bbb-a48e-a178bea06be4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:15:11.366717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:11.366717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:11.366717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:11.366717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:11.366717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:11.366717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:11.366717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:11.366717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:11.366717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:11.366717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:11.366717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:11.366717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:11.366717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:11.366717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:11.366717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:11.367717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:11.367717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:11.367717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:11.367717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:11.367717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:11.367717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:11.367717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:11.367717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:11.367717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:11.367717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:11.367717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:11.367717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:11.367717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:11.367717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:11.367717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:11.367717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:11.368498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:11.368498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:11.368498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:11.368498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:11.368498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:11.368498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:11.368498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:11.368498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:11.368498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:11.368498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:11.368498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:15:11.368498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:11.371888 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103dfce-7575-4ed9-8545-d891364848cd None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:11.375228 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 214/422] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:15:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:15:11.383251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843df7f3-232f-4bbb-a48e-a178bea06be4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:11.462385 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff622f79-f4be-428a-a4fe-f0bf23f407d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:11.463056 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff622f79-f4be-428a-a4fe-f0bf23f407d2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:11.463358 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff622f79-f4be-428a-a4fe-f0bf23f407d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:11.463664 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff622f79-f4be-428a-a4fe-f0bf23f407d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:11.495066 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:11.508235 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843df7f3-232f-4bbb-a48e-a178bea06be4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:11.552418 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce88e2-aba3-4738-99cd-3484dfc31be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:11.554006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce88e2-aba3-4738-99cd-3484dfc31be2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:11.554819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce88e2-aba3-4738-99cd-3484dfc31be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:11.556686 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce88e2-aba3-4738-99cd-3484dfc31be2 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:11.557864 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce88e2-aba3-4738-99cd-3484dfc31be2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:11.558349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce88e2-aba3-4738-99cd-3484dfc31be2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:11.560109 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce88e2-aba3-4738-99cd-3484dfc31be2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:11.570763 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 215/423] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:15:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:11.583172 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce88e2-aba3-4738-99cd-3484dfc31be2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:11.666918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff622f79-f4be-428a-a4fe-f0bf23f407d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:11.668446 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff622f79-f4be-428a-a4fe-f0bf23f407d2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:11.670960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:11.712166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce88e2-aba3-4738-99cd-3484dfc31be2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:11.767789 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c7914-c034-4609-9e90-58392cc48edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:11.769812 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c7914-c034-4609-9e90-58392cc48edb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:11.770708 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c7914-c034-4609-9e90-58392cc48edb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:11.771484 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c7914-c034-4609-9e90-58392cc48edb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:11.842961 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:11.932351 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 209/424] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:15:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 745 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 18:15:11.937338 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c7914-c034-4609-9e90-58392cc48edb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:11.939247 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c7914-c034-4609-9e90-58392cc48edb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:11.939367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff622f79-f4be-428a-a4fe-f0bf23f407d2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:12.113525 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 216/425] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:15:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 18:15:12.116738 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c16f2d-eace-4a93-aa69-82bc19dc64dc None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:12.117095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c16f2d-eace-4a93-aa69-82bc19dc64dc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:12.117386 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c16f2d-eace-4a93-aa69-82bc19dc64dc None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:12.119484 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 217/426] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:12.120602 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c16f2d-eace-4a93-aa69-82bc19dc64dc None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:12.153249 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff622f79-f4be-428a-a4fe-f0bf23f407d2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:12.155424 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:12.217572 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefb376-c371-4310-b5dc-92056ec86f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:12.219201 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefb376-c371-4310-b5dc-92056ec86f4e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:12.219699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefb376-c371-4310-b5dc-92056ec86f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:12.220185 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefb376-c371-4310-b5dc-92056ec86f4e None admin] PATH_INFO: `/v3/projects/d5962edfa0c94d88a1056c1ed5e05058` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:12.222495 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefb376-c371-4310-b5dc-92056ec86f4e None admin] RBAC: Authorizing `identity:get_project(project_id=d5962edfa0c94d88a1056c1ed5e05058)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:12.225620 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefb376-c371-4310-b5dc-92056ec86f4e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:12.230595 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 210/427] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:15:11 2022] GET /identity/v3/projects/d5962edfa0c94d88a1056c1ed5e05058 => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:12.233152 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f50b6-55f1-4d2b-b52d-bc5ab6f19401 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:12.233688 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f50b6-55f1-4d2b-b52d-bc5ab6f19401 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:12.234180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f50b6-55f1-4d2b-b52d-bc5ab6f19401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:12.253855 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:12.272130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c44f50b6-55f1-4d2b-b52d-bc5ab6f19401 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:12.274459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c44f50b6-55f1-4d2b-b52d-bc5ab6f19401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:12.344070 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c16f2d-eace-4a93-aa69-82bc19dc64dc None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:12.402982 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 211/428] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:12.403741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98f021-4ec8-4d11-af80-83c2de46126f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:12.403869 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f50b6-55f1-4d2b-b52d-bc5ab6f19401 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:12.404998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98f021-4ec8-4d11-af80-83c2de46126f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:12.405292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98f021-4ec8-4d11-af80-83c2de46126f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:12.405598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98f021-4ec8-4d11-af80-83c2de46126f None admin] PATH_INFO: `/v3/projects/d5962edfa0c94d88a1056c1ed5e05058` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:12.407175 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de98f021-4ec8-4d11-af80-83c2de46126f None admin] RBAC: Authorizing `identity:get_project(project_id=d5962edfa0c94d88a1056c1ed5e05058)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:12.409038 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de98f021-4ec8-4d11-af80-83c2de46126f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:12.417704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 218/429] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:15:12 2022] GET /identity/v3/projects/d5962edfa0c94d88a1056c1ed5e05058 => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:12.419984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98f021-4ec8-4d11-af80-83c2de46126f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:12.597481 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:12.603539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98f021-4ec8-4d11-af80-83c2de46126f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:12.604865 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f50b6-55f1-4d2b-b52d-bc5ab6f19401 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:12.640121 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:12.661534 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70712198-bd50-4e97-b32d-41c0df11f472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:12.662822 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70712198-bd50-4e97-b32d-41c0df11f472 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:12.663110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70712198-bd50-4e97-b32d-41c0df11f472 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:12.663310 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985787a-2605-467f-b4bf-ed021b0c36bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:12.663427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70712198-bd50-4e97-b32d-41c0df11f472 None admin] PATH_INFO: `/v3/groups/bb43d54bfa084cd59d6ce44ca23cdb96` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:12.664310 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985787a-2605-467f-b4bf-ed021b0c36bd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:12.664602 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985787a-2605-467f-b4bf-ed021b0c36bd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:12.664845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70712198-bd50-4e97-b32d-41c0df11f472 None admin] RBAC: Authorizing `identity:get_group(group_id=bb43d54bfa084cd59d6ce44ca23cdb96)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:12.664993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985787a-2605-467f-b4bf-ed021b0c36bd None admin] PATH_INFO: `/v3/groups/bb43d54bfa084cd59d6ce44ca23cdb96` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:12.666310 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70712198-bd50-4e97-b32d-41c0df11f472 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:12.666679 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985787a-2605-467f-b4bf-ed021b0c36bd None admin] RBAC: Authorizing `identity:get_group(group_id=bb43d54bfa084cd59d6ce44ca23cdb96)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:12.668403 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 219/430] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 30 18:15:12 2022] GET /identity/v3/groups/bb43d54bfa084cd59d6ce44ca23cdb96 => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:12.668762 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985787a-2605-467f-b4bf-ed021b0c36bd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:12.669464 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70712198-bd50-4e97-b32d-41c0df11f472 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:12.677516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 212/431] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 30 18:15:12 2022] GET /identity/v3/groups/bb43d54bfa084cd59d6ce44ca23cdb96 => generated 226 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:12.679000 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985787a-2605-467f-b4bf-ed021b0c36bd None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:12.829463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70712198-bd50-4e97-b32d-41c0df11f472 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:12.835777 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985787a-2605-467f-b4bf-ed021b0c36bd None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:12.891933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21c9a5-7370-4c6a-9a18-756fc347e4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:12.892757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21c9a5-7370-4c6a-9a18-756fc347e4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:12.893254 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21c9a5-7370-4c6a-9a18-756fc347e4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:12.893719 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21c9a5-7370-4c6a-9a18-756fc347e4f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:12.895661 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add28fec-8e12-4b44-95f9-42ed49322f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:12.898108 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add28fec-8e12-4b44-95f9-42ed49322f12 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:12.898533 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add28fec-8e12-4b44-95f9-42ed49322f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:12.898917 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add28fec-8e12-4b44-95f9-42ed49322f12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:12.900027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add28fec-8e12-4b44-95f9-42ed49322f12 None admin] RBAC: Adding query filter params (group.id=bb43d54bfa084cd59d6ce44ca23cdb96, role.id=3fbb71ae2de04e78b68d6d6f407ef09c, scope.project.id=d5962edfa0c94d88a1056c1ed5e05058) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:12.900721 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add28fec-8e12-4b44-95f9-42ed49322f12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:12.903450 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add28fec-8e12-4b44-95f9-42ed49322f12 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:12.910416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21c9a5-7370-4c6a-9a18-756fc347e4f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:12.919369 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 213/432] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 30 18:15:12 2022] GET /identity/v3/role_assignments?group.id=bb43d54bfa084cd59d6ce44ca23cdb96&scope.project.id=d5962edfa0c94d88a1056c1ed5e05058&role.id=3fbb71ae2de04e78b68d6d6f407ef09c => generated 611 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:12.920439 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add28fec-8e12-4b44-95f9-42ed49322f12 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:12.921365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-8e21c9a5-7370-4c6a-9a18-756fc347e4f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:15:12.921365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:12.921365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:12.921365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:12.921365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:12.921365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:12.921365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:12.921365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:12.921365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:12.921365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:12.921365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:12.921365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:12.921365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:12.921365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:12.921365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:12.921365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:12.923040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:12.923040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:12.923040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:12.923040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:12.923040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:12.923040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:12.923040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:12.923040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:12.923040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:12.923040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:12.923040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:12.923040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:12.923040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:12.923040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:12.923040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:12.923040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:12.924388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:12.924388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:12.924388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:12.924388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:12.924388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:12.924388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:12.924388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:12.924388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:12.924388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:12.924388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:12.924388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 18:15:12.924388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:12.927204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 220/433] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:15:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:12.939352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21c9a5-7370-4c6a-9a18-756fc347e4f2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:13.010639 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:13.091007 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:13.167535 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add28fec-8e12-4b44-95f9-42ed49322f12 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:13.194068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21c9a5-7370-4c6a-9a18-756fc347e4f2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:13.220634 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37acc9-eb6b-42b4-806c-2715f2fa379d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:13.221305 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37acc9-eb6b-42b4-806c-2715f2fa379d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:13.221621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37acc9-eb6b-42b4-806c-2715f2fa379d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:13.221901 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37acc9-eb6b-42b4-806c-2715f2fa379d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:13.223284 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab37acc9-eb6b-42b4-806c-2715f2fa379d None admin] RBAC: Adding query filter params (group.id=bb43d54bfa084cd59d6ce44ca23cdb96, role.id=d5f874a3d78f4ee0ade318e1bbebf7ec, scope.project.id=d5962edfa0c94d88a1056c1ed5e05058) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:13.223604 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab37acc9-eb6b-42b4-806c-2715f2fa379d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:13.225479 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab37acc9-eb6b-42b4-806c-2715f2fa379d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:13.234116 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 214/434] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 30 18:15:12 2022] GET /identity/v3/role_assignments?group.id=bb43d54bfa084cd59d6ce44ca23cdb96&scope.project.id=d5962edfa0c94d88a1056c1ed5e05058&role.id=d5f874a3d78f4ee0ade318e1bbebf7ec => generated 611 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:13.242197 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1dbee-08fe-4a77-889e-aebaa04ed3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:13.242459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce045f-aa16-4a30-8fb7-c3871b48f44c None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:13.242694 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce045f-aa16-4a30-8fb7-c3871b48f44c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:13.243162 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce045f-aa16-4a30-8fb7-c3871b48f44c None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:13.243820 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1dbee-08fe-4a77-889e-aebaa04ed3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:13.244306 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1dbee-08fe-4a77-889e-aebaa04ed3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:13.244799 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1dbee-08fe-4a77-889e-aebaa04ed3f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:13.245856 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1dbee-08fe-4a77-889e-aebaa04ed3f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:13.246169 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 215/435] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:13.246683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1dbee-08fe-4a77-889e-aebaa04ed3f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:13.248353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1dbee-08fe-4a77-889e-aebaa04ed3f2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:13.254300 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaddb6c5-fdd4-4d67-9a2a-93ac10d127b3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:13.254825 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaddb6c5-fdd4-4d67-9a2a-93ac10d127b3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:13.255270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaddb6c5-fdd4-4d67-9a2a-93ac10d127b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:13.265371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 221/436] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:15:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:13.275549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1dbee-08fe-4a77-889e-aebaa04ed3f2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:13.283717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-aaddb6c5-fdd4-4d67-9a2a-93ac10d127b3 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:13.285076 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-aaddb6c5-fdd4-4d67-9a2a-93ac10d127b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:13.395269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 216/437] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:13.424415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaddb6c5-fdd4-4d67-9a2a-93ac10d127b3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:13.469898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:13.490251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1dbee-08fe-4a77-889e-aebaa04ed3f2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:13.512391 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:13.566869 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9ebb3-b97a-4f3d-baca-2a9fefc3c8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:13.567641 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9ebb3-b97a-4f3d-baca-2a9fefc3c8cd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:13.567942 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9ebb3-b97a-4f3d-baca-2a9fefc3c8cd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:13.568240 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9ebb3-b97a-4f3d-baca-2a9fefc3c8cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:13.613716 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaddb6c5-fdd4-4d67-9a2a-93ac10d127b3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:13.674321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacbcf76-d419-4b3a-9c30-6a51923bfca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:13.675399 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbcf76-d419-4b3a-9c30-6a51923bfca8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:13.675690 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbcf76-d419-4b3a-9c30-6a51923bfca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:13.675974 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbcf76-d419-4b3a-9c30-6a51923bfca8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:13.691875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacbcf76-d419-4b3a-9c30-6a51923bfca8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:13.704028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-eacbcf76-d419-4b3a-9c30-6a51923bfca8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:15:13.704028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:13.704028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:13.704028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:13.704028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:13.704028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:13.704028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:13.704028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:13.704028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:13.704028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:13.704028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:13.704028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:13.704028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:13.704028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:13.704028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:13.704028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:13.706109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:13.706109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:13.706109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:13.706109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:13.706109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:13.706109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:13.706109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:13.706109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:13.706109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:13.706109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:13.706109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:13.706109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:13.706109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:13.706109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:13.706109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:13.706109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:13.707955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:13.707955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:13.707955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:13.707955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:13.707955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:13.707955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:13.707955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:13.707955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:13.707955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:13.707955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:13.707955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:15:13.707955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:13.707955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 217/438] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:15:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:15:13.707955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81a8f5-abd9-4e83-9bce-93d8584139ac None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:13.710608 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81a8f5-abd9-4e83-9bce-93d8584139ac None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:13.710608 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81a8f5-abd9-4e83-9bce-93d8584139ac None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:13.710608 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 218/439] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:13.721359 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81a8f5-abd9-4e83-9bce-93d8584139ac None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:13.723752 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9ebb3-b97a-4f3d-baca-2a9fefc3c8cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:13.725872 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9ebb3-b97a-4f3d-baca-2a9fefc3c8cd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:13.862648 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:13.888182 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81a8f5-abd9-4e83-9bce-93d8584139ac None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:13.923924 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 222/440] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:15:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 649 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:15:13.932764 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac5fec-96bf-4e84-b052-3bf0dfe82a80 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:13.933301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac5fec-96bf-4e84-b052-3bf0dfe82a80 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:13.933653 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac5fec-96bf-4e84-b052-3bf0dfe82a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:13.953120 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3fac5fec-96bf-4e84-b052-3bf0dfe82a80 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:13.954650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3fac5fec-96bf-4e84-b052-3bf0dfe82a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:13.958353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:13.960717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc74cc-f0c7-47e7-9106-df2b595b5a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:13.962265 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc74cc-f0c7-47e7-9106-df2b595b5a82 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:13.962798 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc74cc-f0c7-47e7-9106-df2b595b5a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:13.963328 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc74cc-f0c7-47e7-9106-df2b595b5a82 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:13.964793 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc74cc-f0c7-47e7-9106-df2b595b5a82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:13.965471 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc74cc-f0c7-47e7-9106-df2b595b5a82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:13.970156 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc74cc-f0c7-47e7-9106-df2b595b5a82 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:13.984353 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 219/441] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:15:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:13.985442 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc74cc-f0c7-47e7-9106-df2b595b5a82 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:14.067016 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 223/442] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:14.068792 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac5fec-96bf-4e84-b052-3bf0dfe82a80 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:14.153390 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc74cc-f0c7-47e7-9106-df2b595b5a82 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:14.211089 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac5fec-96bf-4e84-b052-3bf0dfe82a80 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:14.216565 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac696b-06bd-4a7c-8ea9-b5773bf5c995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:14.217462 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac696b-06bd-4a7c-8ea9-b5773bf5c995 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:14.217840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac696b-06bd-4a7c-8ea9-b5773bf5c995 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:14.218139 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac696b-06bd-4a7c-8ea9-b5773bf5c995 None admin] PATH_INFO: `/v3/projects/9beede2985d54385bc4532fb19a265a5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:14.219652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac696b-06bd-4a7c-8ea9-b5773bf5c995 None admin] RBAC: Authorizing `identity:get_project(project_id=9beede2985d54385bc4532fb19a265a5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:14.221136 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac696b-06bd-4a7c-8ea9-b5773bf5c995 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:14.223111 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 220/443] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:15:13 2022] GET /identity/v3/projects/9beede2985d54385bc4532fb19a265a5 => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:14.224373 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac696b-06bd-4a7c-8ea9-b5773bf5c995 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:14.248628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:14.258153 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284132ee-0da9-4c69-b369-23109257a29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:14.259129 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284132ee-0da9-4c69-b369-23109257a29c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:14.259532 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284132ee-0da9-4c69-b369-23109257a29c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:14.259965 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284132ee-0da9-4c69-b369-23109257a29c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:14.275509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:14.350691 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284132ee-0da9-4c69-b369-23109257a29c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:14.352207 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284132ee-0da9-4c69-b369-23109257a29c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:14.362114 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac696b-06bd-4a7c-8ea9-b5773bf5c995 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:14.410553 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a9ce7-e23c-4b49-9674-d6180b6b91c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:14.411479 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a9ce7-e23c-4b49-9674-d6180b6b91c8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:14.411759 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a9ce7-e23c-4b49-9674-d6180b6b91c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:14.412036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a9ce7-e23c-4b49-9674-d6180b6b91c8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:14.424027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a9ce7-e23c-4b49-9674-d6180b6b91c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:14.432584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d92a9ce7-e23c-4b49-9674-d6180b6b91c8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:15:14.432584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:14.432584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:14.432584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:14.432584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:14.432584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:14.432584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:14.432584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:14.432584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:14.432584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:14.432584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:14.432584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:14.432584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:14.432584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:14.432584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:14.432584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:14.433519 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:14.433519 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:14.433519 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:14.433519 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:14.433519 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:14.433519 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:14.433519 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:14.433519 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:14.433519 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:14.433519 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:14.433519 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:14.433519 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:14.433519 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:14.433519 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:14.433519 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:14.433519 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:14.439178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:14.439178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:14.439178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:14.439178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:14.439178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:14.439178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:14.439178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:14.439178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:14.439178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:14.439178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:14.439178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:15:14.439178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:14.439178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 221/444] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 30 18:15:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:14.439178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a9ce7-e23c-4b49-9674-d6180b6b91c8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:14.483641 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 224/445] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:15:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:15:14.484865 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284132ee-0da9-4c69-b369-23109257a29c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:14.561389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:14.586773 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:14.631900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a9ce7-e23c-4b49-9674-d6180b6b91c8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:14.676768 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284132ee-0da9-4c69-b369-23109257a29c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:14.679964 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd7b9e-7ac7-4a26-98b4-f0a4de11f601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:14.680844 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd7b9e-7ac7-4a26-98b4-f0a4de11f601 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:14.681127 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd7b9e-7ac7-4a26-98b4-f0a4de11f601 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:14.681411 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd7b9e-7ac7-4a26-98b4-f0a4de11f601 None admin] PATH_INFO: `/v3/groups/bb43d54bfa084cd59d6ce44ca23cdb96` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:14.682762 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd7b9e-7ac7-4a26-98b4-f0a4de11f601 None admin] RBAC: Authorizing `identity:get_group(group_id=bb43d54bfa084cd59d6ce44ca23cdb96)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:14.684238 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd7b9e-7ac7-4a26-98b4-f0a4de11f601 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:14.685971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 222/446] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 30 18:15:14 2022] GET /identity/v3/groups/bb43d54bfa084cd59d6ce44ca23cdb96 => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:14.689244 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd7b9e-7ac7-4a26-98b4-f0a4de11f601 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:14.738850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf260de-8c21-4fd1-832d-c47b84ed5215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:14.739757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf260de-8c21-4fd1-832d-c47b84ed5215 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:14.740063 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf260de-8c21-4fd1-832d-c47b84ed5215 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:14.741039 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf260de-8c21-4fd1-832d-c47b84ed5215 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:14.742375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf260de-8c21-4fd1-832d-c47b84ed5215 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:14.742759 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf260de-8c21-4fd1-832d-c47b84ed5215 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:14.744859 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf260de-8c21-4fd1-832d-c47b84ed5215 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:14.761837 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 225/447] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:14.763730 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf260de-8c21-4fd1-832d-c47b84ed5215 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:14.832136 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd7b9e-7ac7-4a26-98b4-f0a4de11f601 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:14.884709 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1afc9-0e75-4448-b41a-e428eba09767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:14.885625 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1afc9-0e75-4448-b41a-e428eba09767 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:14.885897 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1afc9-0e75-4448-b41a-e428eba09767 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:14.886173 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1afc9-0e75-4448-b41a-e428eba09767 None admin] PATH_INFO: `/v3/projects/6ed5e6c375de4a6186ba026da89db56b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:14.887460 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1afc9-0e75-4448-b41a-e428eba09767 None admin] RBAC: Authorizing `identity:get_project(project_id=6ed5e6c375de4a6186ba026da89db56b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:14.888960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1afc9-0e75-4448-b41a-e428eba09767 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:14.890762 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 223/448] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:15:14 2022] GET /identity/v3/projects/6ed5e6c375de4a6186ba026da89db56b => generated 344 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:14.891564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1afc9-0e75-4448-b41a-e428eba09767 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:14.908819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf260de-8c21-4fd1-832d-c47b84ed5215 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:14.909487 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:14.910888 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:14.952143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d407b5-9d2c-42c3-9e4c-a4dfae415934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:14.952833 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d407b5-9d2c-42c3-9e4c-a4dfae415934 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:14.953302 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d407b5-9d2c-42c3-9e4c-a4dfae415934 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:14.953833 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d407b5-9d2c-42c3-9e4c-a4dfae415934 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:14.955229 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d407b5-9d2c-42c3-9e4c-a4dfae415934 None admin] RBAC: Adding query filter params (group.id=bb43d54bfa084cd59d6ce44ca23cdb96, role.id=3fbb71ae2de04e78b68d6d6f407ef09c, scope.project.id=9beede2985d54385bc4532fb19a265a5) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:14.955557 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d407b5-9d2c-42c3-9e4c-a4dfae415934 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:14.957302 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d407b5-9d2c-42c3-9e4c-a4dfae415934 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:14.964799 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 226/449] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 30 18:15:14 2022] GET /identity/v3/role_assignments?group.id=bb43d54bfa084cd59d6ce44ca23cdb96&scope.project.id=9beede2985d54385bc4532fb19a265a5&role.id=3fbb71ae2de04e78b68d6d6f407ef09c => generated 611 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:14.965876 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d407b5-9d2c-42c3-9e4c-a4dfae415934 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:15.031454 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1afc9-0e75-4448-b41a-e428eba09767 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:15.079068 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281abb7-1abb-4808-b874-8a74936c7d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:15.079863 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281abb7-1abb-4808-b874-8a74936c7d6c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:15.080242 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281abb7-1abb-4808-b874-8a74936c7d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:15.080560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281abb7-1abb-4808-b874-8a74936c7d6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:15.143452 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d407b5-9d2c-42c3-9e4c-a4dfae415934 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:15.175002 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7281abb7-1abb-4808-b874-8a74936c7d6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:15.177236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7281abb7-1abb-4808-b874-8a74936c7d6c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:15.197347 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727df8d-199e-48a5-8766-23e039f49cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:15.199832 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727df8d-199e-48a5-8766-23e039f49cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:15.199832 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727df8d-199e-48a5-8766-23e039f49cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:15.199832 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727df8d-199e-48a5-8766-23e039f49cb5 None admin] PATH_INFO: `/v3/groups/05a2807df9df46cd92b4efb85984b78c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:15.202403 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a727df8d-199e-48a5-8766-23e039f49cb5 None admin] RBAC: Authorizing `identity:get_group(group_id=05a2807df9df46cd92b4efb85984b78c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:15.203927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a727df8d-199e-48a5-8766-23e039f49cb5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:15.205910 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 227/450] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 30 18:15:14 2022] GET /identity/v3/groups/05a2807df9df46cd92b4efb85984b78c => generated 229 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:15.212135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:15.213890 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727df8d-199e-48a5-8766-23e039f49cb5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:15.281473 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:15.359623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 224/451] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:15:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:15:15.367664 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281abb7-1abb-4808-b874-8a74936c7d6c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:15.393056 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727df8d-199e-48a5-8766-23e039f49cb5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:15.434689 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09891641-4ce7-4bba-a187-31d339a3a0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:15.436088 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09891641-4ce7-4bba-a187-31d339a3a0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:15.436388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09891641-4ce7-4bba-a187-31d339a3a0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:15.436677 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09891641-4ce7-4bba-a187-31d339a3a0c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:15.437784 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09891641-4ce7-4bba-a187-31d339a3a0c2 None admin] RBAC: Adding query filter params (group.id=05a2807df9df46cd92b4efb85984b78c, role.id=0ef3687af20a443281e3cfdceca3597d, scope.project.id=6ed5e6c375de4a6186ba026da89db56b) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:15.438295 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09891641-4ce7-4bba-a187-31d339a3a0c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:15.440954 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09891641-4ce7-4bba-a187-31d339a3a0c2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:15.449761 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 228/452] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 30 18:15:15 2022] GET /identity/v3/role_assignments?group.id=05a2807df9df46cd92b4efb85984b78c&scope.project.id=6ed5e6c375de4a6186ba026da89db56b&role.id=0ef3687af20a443281e3cfdceca3597d => generated 611 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:15:15.504384 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281abb7-1abb-4808-b874-8a74936c7d6c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:15.534588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:15.574543 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ea64b-ee6d-410d-82a5-a8a132c0ebfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:15.575448 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ea64b-ee6d-410d-82a5-a8a132c0ebfa None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:15.575732 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ea64b-ee6d-410d-82a5-a8a132c0ebfa None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:15.576010 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ea64b-ee6d-410d-82a5-a8a132c0ebfa None admin] PATH_INFO: `/v3/projects/9beede2985d54385bc4532fb19a265a5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:15.577368 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ea64b-ee6d-410d-82a5-a8a132c0ebfa None admin] RBAC: Authorizing `identity:get_project(project_id=9beede2985d54385bc4532fb19a265a5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:15.578978 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ea64b-ee6d-410d-82a5-a8a132c0ebfa None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:15.580885 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 225/453] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:15:15 2022] GET /identity/v3/projects/9beede2985d54385bc4532fb19a265a5 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:15.590519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09891641-4ce7-4bba-a187-31d339a3a0c2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:15.717752 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:15.721710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09891641-4ce7-4bba-a187-31d339a3a0c2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:15.758437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59491065-73ea-4905-a50c-c5602d45f306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:15.759322 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59491065-73ea-4905-a50c-c5602d45f306 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:15.759607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59491065-73ea-4905-a50c-c5602d45f306 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:15.760026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59491065-73ea-4905-a50c-c5602d45f306 None admin] PATH_INFO: `/v3/groups/bb43d54bfa084cd59d6ce44ca23cdb96` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:15.761235 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59491065-73ea-4905-a50c-c5602d45f306 None admin] RBAC: Authorizing `identity:get_group(group_id=bb43d54bfa084cd59d6ce44ca23cdb96)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:15.762722 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59491065-73ea-4905-a50c-c5602d45f306 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:15.764594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 229/454] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 30 18:15:15 2022] GET /identity/v3/groups/bb43d54bfa084cd59d6ce44ca23cdb96 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:15:15.771314 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ea64b-ee6d-410d-82a5-a8a132c0ebfa None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:15.879877 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ea64b-ee6d-410d-82a5-a8a132c0ebfa None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:15.915806 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66322cd4-ed4a-4be5-83a8-d39aa1754d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:15.916465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66322cd4-ed4a-4be5-83a8-d39aa1754d2f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:15.916754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66322cd4-ed4a-4be5-83a8-d39aa1754d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:15.917036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66322cd4-ed4a-4be5-83a8-d39aa1754d2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:15.917909 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66322cd4-ed4a-4be5-83a8-d39aa1754d2f None admin] RBAC: Adding query filter params (group.id=bb43d54bfa084cd59d6ce44ca23cdb96, role.id=d5f874a3d78f4ee0ade318e1bbebf7ec, scope.project.id=9beede2985d54385bc4532fb19a265a5) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:15.918224 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66322cd4-ed4a-4be5-83a8-d39aa1754d2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:15.919978 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66322cd4-ed4a-4be5-83a8-d39aa1754d2f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:15.926387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 226/455] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 30 18:15:15 2022] GET /identity/v3/role_assignments?group.id=bb43d54bfa084cd59d6ce44ca23cdb96&scope.project.id=9beede2985d54385bc4532fb19a265a5&role.id=d5f874a3d78f4ee0ade318e1bbebf7ec => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:18.786020 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c37c78-d069-4589-81c6-f7b489b712de None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:18.786539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c37c78-d069-4589-81c6-f7b489b712de None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:18.786539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c37c78-d069-4589-81c6-f7b489b712de None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:18.788791 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 230/456] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:18.794732 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3dc567-a8f8-4e20-a308-9a9ddf508fb0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:18.795616 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3dc567-a8f8-4e20-a308-9a9ddf508fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:18.796087 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3dc567-a8f8-4e20-a308-9a9ddf508fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:18.813889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4f3dc567-a8f8-4e20-a308-9a9ddf508fb0 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:18.815437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4f3dc567-a8f8-4e20-a308-9a9ddf508fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:18.860008 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:18.891107 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 227/457] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 18:15:18.918632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c37c78-d069-4589-81c6-f7b489b712de None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:18.919940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f2a4c-ffd8-480f-8b10-34f4e34d8f18 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:18.920255 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f2a4c-ffd8-480f-8b10-34f4e34d8f18 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:18.920515 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f2a4c-ffd8-480f-8b10-34f4e34d8f18 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:18.921938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 228/458] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:15:18.928799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b6e4e-90d7-4b33-a53d-df7c0ece2f1c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:18.929077 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b6e4e-90d7-4b33-a53d-df7c0ece2f1c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:18.929361 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b6e4e-90d7-4b33-a53d-df7c0ece2f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:18.965170 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ba9b6e4e-90d7-4b33-a53d-df7c0ece2f1c None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:18.969011 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ba9b6e4e-90d7-4b33-a53d-df7c0ece2f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:19.075229 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c37c78-d069-4589-81c6-f7b489b712de None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:19.120738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 229/459] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:19.134202 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d315774-9f29-4ebe-9f22-3e7d99b00c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:19.135551 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d315774-9f29-4ebe-9f22-3e7d99b00c8a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:19.136101 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d315774-9f29-4ebe-9f22-3e7d99b00c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:19.136577 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d315774-9f29-4ebe-9f22-3e7d99b00c8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:19.150521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d315774-9f29-4ebe-9f22-3e7d99b00c8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:19.152349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d315774-9f29-4ebe-9f22-3e7d99b00c8a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:19.167812 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b6e4e-90d7-4b33-a53d-df7c0ece2f1c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:19.171661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4d315774-9f29-4ebe-9f22-3e7d99b00c8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:19.171661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:19.171661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:19.171661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:19.171661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:19.171661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:19.171661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:19.171661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:19.171661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:19.171661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:19.171661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:19.171661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:19.171661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:19.171661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:19.171661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:19.172519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:19.172519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:19.172519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:19.172519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:19.172519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:19.172519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:19.172519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:19.172519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:19.172519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:19.172519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:19.172519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:19.172519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:19.172519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:19.172519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:19.172519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:19.172519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:19.172519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:19.173349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:19.173349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:19.173349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:19.173349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:19.176555 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 231/460] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:19.184688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d315774-9f29-4ebe-9f22-3e7d99b00c8a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:19.246158 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:19.314624 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d315774-9f29-4ebe-9f22-3e7d99b00c8a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:19.358662 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175602f0-984e-48b6-8824-c62991d77698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:19.360386 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175602f0-984e-48b6-8824-c62991d77698 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:19.360901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175602f0-984e-48b6-8824-c62991d77698 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:19.361377 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175602f0-984e-48b6-8824-c62991d77698 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:19.362467 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175602f0-984e-48b6-8824-c62991d77698 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:19.363303 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175602f0-984e-48b6-8824-c62991d77698 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:19.365164 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175602f0-984e-48b6-8824-c62991d77698 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:19.383112 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b6e4e-90d7-4b33-a53d-df7c0ece2f1c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:19.386321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:19.391823 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 232/461] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:19.399610 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440d6f3-d7ab-4800-94a0-46c18bd13f0f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:19.399978 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440d6f3-d7ab-4800-94a0-46c18bd13f0f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:19.400242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440d6f3-d7ab-4800-94a0-46c18bd13f0f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:19.402190 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 233/462] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:19.402864 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440d6f3-d7ab-4800-94a0-46c18bd13f0f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:19.444928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e398bd42-6ed7-492e-a562-6cdc68aea0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:19.446676 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398bd42-6ed7-492e-a562-6cdc68aea0cc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:19.447302 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398bd42-6ed7-492e-a562-6cdc68aea0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:19.447781 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398bd42-6ed7-492e-a562-6cdc68aea0cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:19.461350 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e398bd42-6ed7-492e-a562-6cdc68aea0cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:19.463254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e398bd42-6ed7-492e-a562-6cdc68aea0cc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:19.487468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e398bd42-6ed7-492e-a562-6cdc68aea0cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:19.487468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:19.487468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:19.487468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:19.487468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:19.487468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:19.487468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:19.487468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:19.487468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:19.487468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:19.487468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:19.487468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:19.487468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:19.487468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:19.487468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:19.488590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:19.488590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:19.488590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:19.488590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:19.488590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:19.488590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:19.488590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:19.488590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:19.488590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:19.488590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:19.488590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:19.488590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:19.488590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:19.488590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:19.488590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:19.488590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:19.488590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:19.489716 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:19.489716 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:19.489716 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:19.489716 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:19.493280 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 230/463] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:15:19.494906 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be164d-3a60-44a6-922a-c8f3b6438340 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:19.495346 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be164d-3a60-44a6-922a-c8f3b6438340 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:19.495757 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be164d-3a60-44a6-922a-c8f3b6438340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:19.501178 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:19.517331 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-48be164d-3a60-44a6-922a-c8f3b6438340 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:19.519270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-48be164d-3a60-44a6-922a-c8f3b6438340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:19.527395 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440d6f3-d7ab-4800-94a0-46c18bd13f0f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:19.572682 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9246e79-032d-4fbd-a688-efd2a36ea38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:19.574178 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9246e79-032d-4fbd-a688-efd2a36ea38e None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:19.574613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9246e79-032d-4fbd-a688-efd2a36ea38e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:19.575064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9246e79-032d-4fbd-a688-efd2a36ea38e None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:19.575955 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9246e79-032d-4fbd-a688-efd2a36ea38e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:19.577953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9246e79-032d-4fbd-a688-efd2a36ea38e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:19.607364 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 231/464] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:19.618044 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be164d-3a60-44a6-922a-c8f3b6438340 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:19.618775 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 234/465] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 30 18:15:19 2022] POST /identity/v3/users => generated 246 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:15:19.654843 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9246e79-032d-4fbd-a688-efd2a36ea38e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:19.787060 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:19.788433 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9246e79-032d-4fbd-a688-efd2a36ea38e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:19.815590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be164d-3a60-44a6-922a-c8f3b6438340 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:19.829775 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e07006-7d35-4dd6-8cb3-e3b258ca91bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:19.830739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e07006-7d35-4dd6-8cb3-e3b258ca91bd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:19.831296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e07006-7d35-4dd6-8cb3-e3b258ca91bd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:19.831599 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e07006-7d35-4dd6-8cb3-e3b258ca91bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:19.834589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:19.846757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e07006-7d35-4dd6-8cb3-e3b258ca91bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:19.848718 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e07006-7d35-4dd6-8cb3-e3b258ca91bd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:19.867278 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbee499-d307-491b-90c9-f59ef5a6a0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:19.867945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbee499-d307-491b-90c9-f59ef5a6a0ce None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:19.868290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbee499-d307-491b-90c9-f59ef5a6a0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:19.868520 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbee499-d307-491b-90c9-f59ef5a6a0ce None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:19.869251 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbee499-d307-491b-90c9-f59ef5a6a0ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:19.869562 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbee499-d307-491b-90c9-f59ef5a6a0ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:19.869816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-22e07006-7d35-4dd6-8cb3-e3b258ca91bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:19.869816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:19.869816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:19.869816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:19.869816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:19.869816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:19.869816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:19.869816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:19.869816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:19.869816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:19.869816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:19.869816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:19.869816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:19.869816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:19.869816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:19.870643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:19.870643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:19.870643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:19.870643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:19.870643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:19.870643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:19.870643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:19.870643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:19.870643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:19.870643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:19.870643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:19.870643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:19.870643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:19.870643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:19.870643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:19.870643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:19.870643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:19.871497 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:19.871497 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:19.871497 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:19.871497 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:19.871704 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbee499-d307-491b-90c9-f59ef5a6a0ce None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:19.871780 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 235/466] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:19.872652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afa57a-4b28-4e45-b263-b31f5a234aaf None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:19.872942 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afa57a-4b28-4e45-b263-b31f5a234aaf None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:19.873219 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afa57a-4b28-4e45-b263-b31f5a234aaf None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:19.881496 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 236/467] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:19.881496 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6afa57a-4b28-4e45-b263-b31f5a234aaf None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:19.883722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 232/468] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:19.885940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1368a-2b47-46ed-af63-8adcb65b13cc None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:19.886237 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1368a-2b47-46ed-af63-8adcb65b13cc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:19.886520 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1368a-2b47-46ed-af63-8adcb65b13cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:19.905312 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d3c1368a-2b47-46ed-af63-8adcb65b13cc None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:19.906569 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d3c1368a-2b47-46ed-af63-8adcb65b13cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:19.990476 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 233/469] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:19.997711 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1368a-2b47-46ed-af63-8adcb65b13cc None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:20.019396 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6afa57a-4b28-4e45-b263-b31f5a234aaf None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:20.064190 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3ec05-cdd0-4056-8e57-e593c6d988c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:20.064878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3ec05-cdd0-4056-8e57-e593c6d988c2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:20.065169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3ec05-cdd0-4056-8e57-e593c6d988c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:20.065458 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3ec05-cdd0-4056-8e57-e593c6d988c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:20.066709 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3ec05-cdd0-4056-8e57-e593c6d988c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:20.067031 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3ec05-cdd0-4056-8e57-e593c6d988c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:20.069286 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3ec05-cdd0-4056-8e57-e593c6d988c2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:20.087255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 237/470] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:15:20.090274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3ec05-cdd0-4056-8e57-e593c6d988c2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:20.114305 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:20.145101 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1368a-2b47-46ed-af63-8adcb65b13cc None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:20.154712 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:20.184776 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441c829-7faf-4b21-bb76-510f2fa34682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:20.185904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441c829-7faf-4b21-bb76-510f2fa34682 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:20.186390 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441c829-7faf-4b21-bb76-510f2fa34682 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:20.186875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441c829-7faf-4b21-bb76-510f2fa34682 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:20.187784 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441c829-7faf-4b21-bb76-510f2fa34682 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:20.189472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441c829-7faf-4b21-bb76-510f2fa34682 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:20.207519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3ec05-cdd0-4056-8e57-e593c6d988c2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:20.225615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 234/471] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 30 18:15:19 2022] POST /identity/v3/users => generated 248 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:15:20.225615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441c829-7faf-4b21-bb76-510f2fa34682 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:20.281233 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47cd02-c4d0-4123-aa00-be11e7876f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:20.282140 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47cd02-c4d0-4123-aa00-be11e7876f2e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:20.282421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47cd02-c4d0-4123-aa00-be11e7876f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:20.282695 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47cd02-c4d0-4123-aa00-be11e7876f2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:20.305738 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47cd02-c4d0-4123-aa00-be11e7876f2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:20.311573 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47cd02-c4d0-4123-aa00-be11e7876f2e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:20.345253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3f47cd02-c4d0-4123-aa00-be11e7876f2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:20.345253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:20.345253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:20.345253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:20.345253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:20.345253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:20.345253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:20.345253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:20.345253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:20.345253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:20.345253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:20.345253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:20.345253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:20.345253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:20.345253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:20.346917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:20.346917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:20.346917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:20.346917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:20.346917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:20.346917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:20.346917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:20.346917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:20.346917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:20.346917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:20.346917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:20.346917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:20.346917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:20.346917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:20.346917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:20.346917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:20.346917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:20.348536 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:20.348536 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:20.348536 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:20.348536 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:20.348536 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 238/472] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:20.348803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6999c-6397-4b47-beab-619a54bc662d None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:20.348978 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6999c-6397-4b47-beab-619a54bc662d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:20.349243 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6999c-6397-4b47-beab-619a54bc662d None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:20.356602 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 239/473] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:20.357523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f6999c-6397-4b47-beab-619a54bc662d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:20.411269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441c829-7faf-4b21-bb76-510f2fa34682 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:20.451285 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:20.459610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c7489-2145-48ed-bf2c-d5eab9fe1665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:20.461508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c7489-2145-48ed-bf2c-d5eab9fe1665 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:20.461830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c7489-2145-48ed-bf2c-d5eab9fe1665 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:20.462419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c7489-2145-48ed-bf2c-d5eab9fe1665 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:20.463334 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c7489-2145-48ed-bf2c-d5eab9fe1665 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:20.466260 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c7489-2145-48ed-bf2c-d5eab9fe1665 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:20.469922 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:20.475607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f6999c-6397-4b47-beab-619a54bc662d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:20.500700 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 235/474] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 30 18:15:20 2022] POST /identity/v3/users => generated 248 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:15:20.502257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8ea9f-cf3a-4b2c-a0c0-b5a096c184e7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:20.503260 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8ea9f-cf3a-4b2c-a0c0-b5a096c184e7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:20.503260 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8ea9f-cf3a-4b2c-a0c0-b5a096c184e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:20.526891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-36b8ea9f-cf3a-4b2c-a0c0-b5a096c184e7 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:20.528513 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-36b8ea9f-cf3a-4b2c-a0c0-b5a096c184e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:20.535988 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a77e8b-10db-479b-903d-127cc65cbb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:20.537170 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a77e8b-10db-479b-903d-127cc65cbb30 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:20.538440 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a77e8b-10db-479b-903d-127cc65cbb30 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:20.538968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a77e8b-10db-479b-903d-127cc65cbb30 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:20.540690 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a77e8b-10db-479b-903d-127cc65cbb30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:20.541221 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a77e8b-10db-479b-903d-127cc65cbb30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:20.543687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a77e8b-10db-479b-903d-127cc65cbb30 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:20.570140 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 240/475] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:20.576271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a77e8b-10db-479b-903d-127cc65cbb30 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:20.653246 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 236/476] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:20.699212 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8ea9f-cf3a-4b2c-a0c0-b5a096c184e7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:20.790059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a77e8b-10db-479b-903d-127cc65cbb30 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:20.853476 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdab00a-3361-416d-b303-e25830490c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:20.854983 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdab00a-3361-416d-b303-e25830490c1c None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:20.855281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdab00a-3361-416d-b303-e25830490c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:20.855560 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdab00a-3361-416d-b303-e25830490c1c None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:20.856611 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdab00a-3361-416d-b303-e25830490c1c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:20.859151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdab00a-3361-416d-b303-e25830490c1c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:20.870325 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8ea9f-cf3a-4b2c-a0c0-b5a096c184e7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:20.893697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 241/477] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 30 18:15:20 2022] POST /identity/v3/users => generated 249 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:15:20.913363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:20.923674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61792bd5-bc8e-49ca-a6f7-03d80a359044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:20.924562 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61792bd5-bc8e-49ca-a6f7-03d80a359044 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:20.924918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61792bd5-bc8e-49ca-a6f7-03d80a359044 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:20.925260 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61792bd5-bc8e-49ca-a6f7-03d80a359044 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:20.926281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61792bd5-bc8e-49ca-a6f7-03d80a359044 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:20.928543 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61792bd5-bc8e-49ca-a6f7-03d80a359044 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:20.937722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.sql.core [None req-61792bd5-bc8e-49ca-a6f7-03d80a359044 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 18:15:20.937722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 30 18:15:20.937722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [parameters: {'id': 'e607240693bd45e6b76aea9c33b4aeb8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 30 18:15:20.937722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80206) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 18:15:20.956940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-61792bd5-bc8e-49ca-a6f7-03d80a359044 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 30 18:15:20.956940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:20.956940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 18:15:20.956940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 18:15:20.956940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 18:15:20.956940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 18:15:20.956940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 18:15:20.956940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 18:15:20.956940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 18:15:20.956940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application conn.query(q) Jun 30 18:15:20.956940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 18:15:20.956940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 18:15:20.956940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 18:15:20.956940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result.read() Jun 30 18:15:20.956940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 18:15:20.957920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 18:15:20.957920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 18:15:20.957920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 18:15:20.957920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 18:15:20.957920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 18:15:20.957920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 18:15:20.957920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 18:15:20.957920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 18:15:20.957920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:20.957920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 18:15:20.957920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:20.957920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:20.957920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 18:15:20.957920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 18:15:20.957920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 30 18:15:20.957920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return ref.to_dict() Jun 30 18:15:20.957920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 18:15:20.957920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application next(self.gen) Jun 30 18:15:20.959321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 18:15:20.959321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application yield resource Jun 30 18:15:20.959321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 18:15:20.959321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application next(self.gen) Jun 30 18:15:20.959321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 18:15:20.959321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 18:15:20.959321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 18:15:20.959321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 18:15:20.959321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 18:15:20.959321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise self.value Jun 30 18:15:20.959321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 18:15:20.959321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 18:15:20.959321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 18:15:20.959321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application session.commit() Jun 30 18:15:20.959321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 18:15:20.959321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 18:15:20.959321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 18:15:20.960311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 18:15:20.960311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 18:15:20.960311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.session.flush() Jun 30 18:15:20.960311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 18:15:20.960311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 18:15:20.960311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 18:15:20.960311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 18:15:20.960311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 18:15:20.960311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application compat.raise_( Jun 30 18:15:20.960311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 18:15:20.960311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception Jun 30 18:15:20.960311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 18:15:20.960311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 18:15:20.960311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 18:15:20.960311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 18:15:20.960311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 18:15:20.960311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 18:15:20.961717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 18:15:20.961717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 18:15:20.961717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 30 18:15:20.961717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 30 18:15:20.961717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 18:15:20.961717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 18:15:20.961717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 18:15:20.961717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 18:15:20.961717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 18:15:20.961717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 18:15:20.961717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 18:15:20.961717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 18:15:20.961717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 18:15:20.961717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 18:15:20.961717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 18:15:20.961717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception Jun 30 18:15:20.963756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 18:15:20.963756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 18:15:20.963756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 18:15:20.963756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 18:15:20.963756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 18:15:20.963756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 18:15:20.963756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 18:15:20.963756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application conn.query(q) Jun 30 18:15:20.963756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 18:15:20.963756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 18:15:20.963756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 18:15:20.963756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result.read() Jun 30 18:15:20.963756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 18:15:20.963756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 18:15:20.963756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 18:15:20.963756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 18:15:20.968430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 18:15:20.968430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 18:15:20.968430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 18:15:20.968430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 18:15:20.968430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 18:15:20.968430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 30 18:15:20.968430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [parameters: {'id': 'e607240693bd45e6b76aea9c33b4aeb8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 30 18:15:20.968430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 18:15:20.968430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:20.968430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:15:20.968430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:20.968430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:20.968430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:20.968430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:20.968430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:20.968430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:20.971810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:20.971810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:20.971810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:20.971810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:20.971810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:20.971810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:20.971810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 30 18:15:20.971810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 30 18:15:20.971810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:20.971810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:20.971810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 30 18:15:20.971810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 30 18:15:20.971810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 18:15:20.971810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 18:15:20.971810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 30 18:15:20.971810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:20.973900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 237/478] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Jun 30 18:15:20 2022] POST /identity/v3/roles => generated 144 bytes in 260 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:15:20.974040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdab00a-3361-416d-b303-e25830490c1c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:20.986778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:21.149950 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdab00a-3361-416d-b303-e25830490c1c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:21.199361 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09006d93-c4e3-4714-9dc7-8ff28d3ea529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:21.200770 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09006d93-c4e3-4714-9dc7-8ff28d3ea529 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:21.201218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09006d93-c4e3-4714-9dc7-8ff28d3ea529 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:21.201609 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09006d93-c4e3-4714-9dc7-8ff28d3ea529 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:21.217352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09006d93-c4e3-4714-9dc7-8ff28d3ea529 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:21.231300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-09006d93-c4e3-4714-9dc7-8ff28d3ea529 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:15:21.231300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:21.231300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:21.231300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:21.231300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:21.231300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:21.231300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:21.231300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:21.231300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:21.231300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:21.231300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:21.231300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:21.231300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:21.231300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:21.231300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:21.231300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:21.232249 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:21.232249 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:21.232249 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:21.232249 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:21.232249 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:21.232249 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:21.232249 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:21.232249 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:21.232249 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:21.232249 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:21.232249 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:21.232249 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:21.232249 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:21.232249 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:21.232249 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:21.232249 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:21.233043 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:21.233043 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:21.233043 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:21.233043 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:21.233043 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:21.233043 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:21.233043 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:21.233043 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:21.233043 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:21.233043 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:21.233043 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 18:15:21.233043 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:21.234227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 242/479] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 30 18:15:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:15:21.239732 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61792bd5-bc8e-49ca-a6f7-03d80a359044 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:21.393616 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61792bd5-bc8e-49ca-a6f7-03d80a359044 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:21.431575 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbecfd-06ec-4312-b2cf-8d8207e05fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:21.432260 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbecfd-06ec-4312-b2cf-8d8207e05fed None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:21.432551 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbecfd-06ec-4312-b2cf-8d8207e05fed None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:21.432833 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbecfd-06ec-4312-b2cf-8d8207e05fed None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:21.433650 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bbecfd-06ec-4312-b2cf-8d8207e05fed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:21.433972 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bbecfd-06ec-4312-b2cf-8d8207e05fed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:21.435393 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bbecfd-06ec-4312-b2cf-8d8207e05fed None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:21.446462 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 238/480] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:22.190670 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc86a3b-296c-4ce1-94ab-e963ca8eaf47 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:22.191216 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc86a3b-296c-4ce1-94ab-e963ca8eaf47 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:22.191290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc86a3b-296c-4ce1-94ab-e963ca8eaf47 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:22.193194 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 243/481] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:22.200918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf4396-dfc5-4b83-a75b-090ed22ac5da None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:22.201430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf4396-dfc5-4b83-a75b-090ed22ac5da None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:22.201996 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf4396-dfc5-4b83-a75b-090ed22ac5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:22.221910 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-acbf4396-dfc5-4b83-a75b-090ed22ac5da None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:22.223141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-acbf4396-dfc5-4b83-a75b-090ed22ac5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:22.268803 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:22.309785 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 239/482] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:22.328942 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc86a3b-296c-4ce1-94ab-e963ca8eaf47 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:22.432172 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:22.478342 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc86a3b-296c-4ce1-94ab-e963ca8eaf47 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:22.531892 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857d226-2ff8-4764-b825-c209c4bd4905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:22.532731 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857d226-2ff8-4764-b825-c209c4bd4905 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:22.533021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857d226-2ff8-4764-b825-c209c4bd4905 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:22.533302 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857d226-2ff8-4764-b825-c209c4bd4905 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:22.548078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b857d226-2ff8-4764-b825-c209c4bd4905 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:22.560274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b857d226-2ff8-4764-b825-c209c4bd4905 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:22.560274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:22.560274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:22.560274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:22.560274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:22.560274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:22.560274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:22.560274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:22.560274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:22.560274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:22.560274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:22.560274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:22.560274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:22.560274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:22.560274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:22.560274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:22.561275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:22.561275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:22.561275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:22.561275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:22.561275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:22.561275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:22.561275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:22.561275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:22.561275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:22.561275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:22.561275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:22.561275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:22.561275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:22.561275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:22.561275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:22.561275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:22.562102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:22.562102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:22.562102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:22.562102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:22.562102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:22.562102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:22.562102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:22.562102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:22.562102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:22.562102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:22.562102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:22.562102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:22.562102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 244/483] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 30 18:15:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:22.567537 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf4396-dfc5-4b83-a75b-090ed22ac5da None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:22.758703 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf4396-dfc5-4b83-a75b-090ed22ac5da None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:22.815333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1246979-1fcb-49ca-a510-ec32a1a32cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:22.815941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1246979-1fcb-49ca-a510-ec32a1a32cda None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:22.816226 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1246979-1fcb-49ca-a510-ec32a1a32cda None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:22.816512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1246979-1fcb-49ca-a510-ec32a1a32cda None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:22.817276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1246979-1fcb-49ca-a510-ec32a1a32cda None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:22.817601 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1246979-1fcb-49ca-a510-ec32a1a32cda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:22.819162 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1246979-1fcb-49ca-a510-ec32a1a32cda None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:22.829704 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 240/484] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 30 18:15:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:22.836090 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857d226-2ff8-4764-b825-c209c4bd4905 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:23.022211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857d226-2ff8-4764-b825-c209c4bd4905 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:23.083768 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb32cf-0ea9-43fe-a3f0-b6d0d1ccb0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:23.085037 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb32cf-0ea9-43fe-a3f0-b6d0d1ccb0fd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:23.085328 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb32cf-0ea9-43fe-a3f0-b6d0d1ccb0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:23.085633 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb32cf-0ea9-43fe-a3f0-b6d0d1ccb0fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:23.097958 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:23.103451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb32cf-0ea9-43fe-a3f0-b6d0d1ccb0fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:23.103621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c285659-8bd2-44d1-96f1-b22201c2f125 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:23.103837 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c285659-8bd2-44d1-96f1-b22201c2f125 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:23.104127 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c285659-8bd2-44d1-96f1-b22201c2f125 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:23.105008 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb32cf-0ea9-43fe-a3f0-b6d0d1ccb0fd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:23.105790 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 241/485] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:23.112925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c152eef-375c-4e8b-b5cd-4c66c0ad3a47 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:23.113220 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c152eef-375c-4e8b-b5cd-4c66c0ad3a47 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:23.113731 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c152eef-375c-4e8b-b5cd-4c66c0ad3a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:23.139022 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-badb32cf-0ea9-43fe-a3f0-b6d0d1ccb0fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:23.139022 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:23.139022 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:23.139022 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:23.139022 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:23.139022 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:23.139022 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:23.139022 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:23.139022 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:23.139022 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:23.139022 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:23.139022 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:23.139022 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:23.139022 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:23.139022 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:23.140095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:23.140095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:23.140095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:23.140095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:23.140095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:23.140095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:23.140095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:23.140095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:23.140095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:23.140095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:23.140095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:23.140095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:23.140095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:23.140095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:23.140095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:23.140095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:23.140095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:23.141036 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:23.141036 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:23.141036 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:23.141036 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:23.141036 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 245/486] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:23.145978 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb32cf-0ea9-43fe-a3f0-b6d0d1ccb0fd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:23.154743 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3c152eef-375c-4e8b-b5cd-4c66c0ad3a47 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:23.155976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3c152eef-375c-4e8b-b5cd-4c66c0ad3a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:23.234876 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:23.274392 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb32cf-0ea9-43fe-a3f0-b6d0d1ccb0fd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:23.321940 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c67be1-407c-473c-82c9-2372f9e74f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:23.322626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c67be1-407c-473c-82c9-2372f9e74f6d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:23.322912 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c67be1-407c-473c-82c9-2372f9e74f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:23.323195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c67be1-407c-473c-82c9-2372f9e74f6d None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:23.323934 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c67be1-407c-473c-82c9-2372f9e74f6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:23.324246 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c67be1-407c-473c-82c9-2372f9e74f6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:23.325669 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c67be1-407c-473c-82c9-2372f9e74f6d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:23.345979 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 246/487] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:23.347269 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c67be1-407c-473c-82c9-2372f9e74f6d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:23.379930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 242/488] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 18:15:23.436829 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:23.446282 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c152eef-375c-4e8b-b5cd-4c66c0ad3a47 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:23.568437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c67be1-407c-473c-82c9-2372f9e74f6d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:23.617440 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95e9f9-7515-4fb9-a5ed-44e6dca0cef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:23.618484 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95e9f9-7515-4fb9-a5ed-44e6dca0cef7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:23.618776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95e9f9-7515-4fb9-a5ed-44e6dca0cef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:23.619059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95e9f9-7515-4fb9-a5ed-44e6dca0cef7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:23.624667 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c152eef-375c-4e8b-b5cd-4c66c0ad3a47 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:23.628635 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95e9f9-7515-4fb9-a5ed-44e6dca0cef7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:23.632555 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95e9f9-7515-4fb9-a5ed-44e6dca0cef7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:23.656688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5a95e9f9-7515-4fb9-a5ed-44e6dca0cef7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 18:15:23.656688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:23.656688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:23.656688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:23.656688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:23.656688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:23.656688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:23.656688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:23.656688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:23.656688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:23.656688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:23.656688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:23.656688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:15:23.656688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:15:23.656688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:15:23.656688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:15:23.657726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:23.657726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:23.657726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:15:23.657726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:23.657726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:15:23.657726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:23.657726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:23.657726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:23.657726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:23.657726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:23.657726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:23.657726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:23.657726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:23.657726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:23.657726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:23.657726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:23.657726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:23.657726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:23.658649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:23.658649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:23.658649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:15:23.658649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:15:23.658649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:15:23.658649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:15:23.658649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:15:23.658649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:15:23.658649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 18:15:23.658649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:23.658649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 247/489] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 30 18:15:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:23.659729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1e587-2489-4efa-8aa4-180e1aaf7bb9 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:23.665625 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1e587-2489-4efa-8aa4-180e1aaf7bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:23.665625 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1e587-2489-4efa-8aa4-180e1aaf7bb9 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:23.666061 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 248/490] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:23.667600 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1e587-2489-4efa-8aa4-180e1aaf7bb9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:23.679695 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca62e0-95d9-4c81-b58c-704d4ed8cfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:23.681263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca62e0-95d9-4c81-b58c-704d4ed8cfd7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:23.681551 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca62e0-95d9-4c81-b58c-704d4ed8cfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:23.681862 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca62e0-95d9-4c81-b58c-704d4ed8cfd7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:23.697542 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca62e0-95d9-4c81-b58c-704d4ed8cfd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:23.708861 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-32ca62e0-95d9-4c81-b58c-704d4ed8cfd7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:23.708861 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:23.708861 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:23.708861 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:23.708861 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:23.708861 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:23.708861 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:23.708861 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:23.708861 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:23.708861 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:23.708861 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:23.708861 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:23.708861 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:23.708861 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:23.708861 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:23.708861 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:23.711948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:23.711948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:23.711948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:23.711948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:23.711948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:23.711948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:23.711948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:23.711948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:23.711948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:23.711948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:23.711948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:23.711948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:23.711948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:23.711948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:23.711948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:23.711948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:23.713936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:23.713936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:23.713936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:23.713936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:23.713936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:23.713936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:23.713936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:23.713936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:23.713936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:23.713936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:23.713936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:23.713936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:23.713936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 243/491] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 30 18:15:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:23.713936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985176b5-0131-4056-9672-43429fc27288 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:23.715722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985176b5-0131-4056-9672-43429fc27288 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:23.715722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985176b5-0131-4056-9672-43429fc27288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:23.735087 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-985176b5-0131-4056-9672-43429fc27288 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:23.736604 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-985176b5-0131-4056-9672-43429fc27288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:23.769803 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:23.802321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1e587-2489-4efa-8aa4-180e1aaf7bb9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:23.815661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:23.845981 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 244/492] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:23.847501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985176b5-0131-4056-9672-43429fc27288 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:23.853690 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e16a78-00cb-46fb-9b3f-cdaf99606f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:23.853690 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e16a78-00cb-46fb-9b3f-cdaf99606f8c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:23.853690 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e16a78-00cb-46fb-9b3f-cdaf99606f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:23.853690 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e16a78-00cb-46fb-9b3f-cdaf99606f8c None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:23.854208 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e16a78-00cb-46fb-9b3f-cdaf99606f8c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:23.854528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e16a78-00cb-46fb-9b3f-cdaf99606f8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:23.857169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e16a78-00cb-46fb-9b3f-cdaf99606f8c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:23.901742 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 249/493] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 30 18:15:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:23.903461 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310afa5-0bed-4548-9a7b-206f078414b9 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:23.903937 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310afa5-0bed-4548-9a7b-206f078414b9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:23.904409 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310afa5-0bed-4548-9a7b-206f078414b9 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:23.907056 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 250/494] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:23.908055 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310afa5-0bed-4548-9a7b-206f078414b9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:24.010254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985176b5-0131-4056-9672-43429fc27288 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:24.046003 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310afa5-0bed-4548-9a7b-206f078414b9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:24.062660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85614c5-76e3-404c-8a2c-d62e6de99bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:24.063442 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85614c5-76e3-404c-8a2c-d62e6de99bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:24.063730 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85614c5-76e3-404c-8a2c-d62e6de99bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:24.064008 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85614c5-76e3-404c-8a2c-d62e6de99bb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:24.064769 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85614c5-76e3-404c-8a2c-d62e6de99bb7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:24.065078 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85614c5-76e3-404c-8a2c-d62e6de99bb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:24.066511 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85614c5-76e3-404c-8a2c-d62e6de99bb7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:24.076355 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 245/495] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 30 18:15:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:24.077197 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85614c5-76e3-404c-8a2c-d62e6de99bb7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:24.092701 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ea8b4-56ce-4735-95e4-4d96b39f3801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:24.093637 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ea8b4-56ce-4735-95e4-4d96b39f3801 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:24.094002 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ea8b4-56ce-4735-95e4-4d96b39f3801 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:24.094334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ea8b4-56ce-4735-95e4-4d96b39f3801 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:24.106468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:24.107660 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ea8b4-56ce-4735-95e4-4d96b39f3801 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:24.116594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-755ea8b4-56ce-4735-95e4-4d96b39f3801 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:24.116594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:24.116594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:24.116594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:24.116594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:24.116594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:24.116594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:24.116594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:24.116594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:24.116594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:24.116594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:24.116594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:24.116594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:24.116594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:24.116594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:24.116594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:24.117623 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:24.117623 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:24.117623 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:24.117623 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:24.117623 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:24.117623 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:24.117623 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:24.117623 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:24.117623 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:24.117623 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:24.117623 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:24.117623 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:24.117623 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:24.117623 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:24.117623 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:24.117623 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:24.118448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:24.118448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:24.118448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:24.118448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:24.118448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:24.118448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:24.118448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:24.118448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:24.118448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:24.118448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:24.118448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:24.118448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:24.118448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 251/496] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 30 18:15:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:24.119643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724bea92-4bec-4940-9d24-16c2667e7c8f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:24.119643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724bea92-4bec-4940-9d24-16c2667e7c8f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:24.119643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724bea92-4bec-4940-9d24-16c2667e7c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:24.138550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-724bea92-4bec-4940-9d24-16c2667e7c8f None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:24.139823 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-724bea92-4bec-4940-9d24-16c2667e7c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:24.159697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:24.220437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 252/497] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:24.221925 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724bea92-4bec-4940-9d24-16c2667e7c8f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:24.255553 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85614c5-76e3-404c-8a2c-d62e6de99bb7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:24.300946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d314a0f-b70c-4da0-a239-ea1e229a67b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:24.301692 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d314a0f-b70c-4da0-a239-ea1e229a67b4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:24.301995 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d314a0f-b70c-4da0-a239-ea1e229a67b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:24.302276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d314a0f-b70c-4da0-a239-ea1e229a67b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:24.358363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724bea92-4bec-4940-9d24-16c2667e7c8f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:24.376365 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d314a0f-b70c-4da0-a239-ea1e229a67b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:24.377893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d314a0f-b70c-4da0-a239-ea1e229a67b4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:24.395192 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fcaefb-ee14-43aa-b52e-27acdfad1931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:24.395989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fcaefb-ee14-43aa-b52e-27acdfad1931 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:24.396281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fcaefb-ee14-43aa-b52e-27acdfad1931 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:24.396565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fcaefb-ee14-43aa-b52e-27acdfad1931 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:24.398574 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:24.411688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fcaefb-ee14-43aa-b52e-27acdfad1931 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:24.414428 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fcaefb-ee14-43aa-b52e-27acdfad1931 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:24.444818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-15fcaefb-ee14-43aa-b52e-27acdfad1931 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:24.444818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:24.444818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:24.444818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:24.444818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:24.444818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:24.444818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:24.444818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:24.444818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:24.444818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:24.444818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:24.444818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:24.444818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:24.444818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:24.444818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:24.445817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:24.445817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:24.445817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:24.445817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:24.445817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:24.445817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:24.445817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:24.445817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:24.445817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:24.445817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:24.445817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:24.445817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:24.445817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:24.445817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:24.445817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:24.445817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:24.445817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:24.446760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:24.446760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:24.446760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:24.446760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:24.446760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 253/498] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:24.448015 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fcaefb-ee14-43aa-b52e-27acdfad1931 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:24.460640 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:24.509779 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 246/499] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:15:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:15:24.510619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d314a0f-b70c-4da0-a239-ea1e229a67b4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:24.565998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fcaefb-ee14-43aa-b52e-27acdfad1931 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:24.600967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f924789-ed5d-4c01-b4b6-b360279f990d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:24.601670 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f924789-ed5d-4c01-b4b6-b360279f990d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:24.601973 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f924789-ed5d-4c01-b4b6-b360279f990d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:24.602253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f924789-ed5d-4c01-b4b6-b360279f990d None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:24.603006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f924789-ed5d-4c01-b4b6-b360279f990d None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:24.603320 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f924789-ed5d-4c01-b4b6-b360279f990d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:24.604700 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f924789-ed5d-4c01-b4b6-b360279f990d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:24.613274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 254/500] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 30 18:15:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:24.614587 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e70bd-00d4-43d1-89ff-765698aefe5f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:24.614882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e70bd-00d4-43d1-89ff-765698aefe5f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:24.616011 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e70bd-00d4-43d1-89ff-765698aefe5f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:24.617049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 255/501] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:24.617785 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e70bd-00d4-43d1-89ff-765698aefe5f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:24.620998 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d314a0f-b70c-4da0-a239-ea1e229a67b4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:24.643476 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:24.658493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f29dc9-53a1-409e-99e7-ab5ae8e33b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:24.659311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f29dc9-53a1-409e-99e7-ab5ae8e33b1e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:24.659606 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f29dc9-53a1-409e-99e7-ab5ae8e33b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:24.659913 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f29dc9-53a1-409e-99e7-ab5ae8e33b1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:24.671836 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f29dc9-53a1-409e-99e7-ab5ae8e33b1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:24.680080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-12f29dc9-53a1-409e-99e7-ab5ae8e33b1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:24.680080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:24.680080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:24.680080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:24.680080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:24.680080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:24.680080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:24.680080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:24.680080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:24.680080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:24.680080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:24.680080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:24.680080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:24.680080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:24.680080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:24.680080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:24.680988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:24.680988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:24.680988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:24.680988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:24.680988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:24.680988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:24.680988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:24.680988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:24.680988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:24.680988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:24.680988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:24.680988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:24.680988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:24.680988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:24.680988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:24.680988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:24.681817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:24.681817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:24.681817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:24.681817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:24.681817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:24.681817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:24.681817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:24.681817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:24.681817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:24.681817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:24.681817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:24.681817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:24.681817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 247/502] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 30 18:15:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:24.682458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f29dc9-53a1-409e-99e7-ab5ae8e33b1e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:24.697807 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:24.736637 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e70bd-00d4-43d1-89ff-765698aefe5f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:24.776314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c155146-a148-452b-829d-8b713dd8700d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:24.776985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c155146-a148-452b-829d-8b713dd8700d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:24.777272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c155146-a148-452b-829d-8b713dd8700d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:24.777551 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c155146-a148-452b-829d-8b713dd8700d None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:24.778310 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c155146-a148-452b-829d-8b713dd8700d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:24.778619 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c155146-a148-452b-829d-8b713dd8700d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:24.780019 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c155146-a148-452b-829d-8b713dd8700d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:24.792857 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 256/503] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:24.793717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c155146-a148-452b-829d-8b713dd8700d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:24.805031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f29dc9-53a1-409e-99e7-ab5ae8e33b1e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:24.839101 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2719efe-3ff7-451d-9739-53f8d26aedf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:24.839895 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2719efe-3ff7-451d-9739-53f8d26aedf9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:24.840181 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2719efe-3ff7-451d-9739-53f8d26aedf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:24.840457 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2719efe-3ff7-451d-9739-53f8d26aedf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:24.852019 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2719efe-3ff7-451d-9739-53f8d26aedf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:24.854876 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2719efe-3ff7-451d-9739-53f8d26aedf9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:24.873904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a2719efe-3ff7-451d-9739-53f8d26aedf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:24.873904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:24.873904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:24.873904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:24.873904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:24.873904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:24.873904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:24.873904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:24.873904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:24.873904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:24.873904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:24.873904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:24.873904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:24.873904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:24.873904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:24.874797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:24.874797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:24.874797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:24.874797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:24.874797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:24.874797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:24.874797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:24.874797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:24.874797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:24.874797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:24.874797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:24.874797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:24.874797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:24.874797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:24.874797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:24.874797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:24.874797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:24.875802 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:24.875802 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:24.875802 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:24.875802 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:24.875802 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 248/504] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:24.876298 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41b778-4397-431b-b145-d8307e2989ae None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:24.876582 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41b778-4397-431b-b145-d8307e2989ae None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:24.876858 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41b778-4397-431b-b145-d8307e2989ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:24.893163 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ef41b778-4397-431b-b145-d8307e2989ae None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:24.894387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ef41b778-4397-431b-b145-d8307e2989ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:24.899380 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c155146-a148-452b-829d-8b713dd8700d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:24.918963 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:24.926492 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:24.935580 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55abcf-1dc1-42f1-bacb-9c31f71be144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:24.936688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55abcf-1dc1-42f1-bacb-9c31f71be144 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:24.936688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55abcf-1dc1-42f1-bacb-9c31f71be144 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:24.936903 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55abcf-1dc1-42f1-bacb-9c31f71be144 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:24.947020 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55abcf-1dc1-42f1-bacb-9c31f71be144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:24.948479 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55abcf-1dc1-42f1-bacb-9c31f71be144 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:24.963283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 249/505] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:24.964120 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41b778-4397-431b-b145-d8307e2989ae None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:24.966251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ad55abcf-1dc1-42f1-bacb-9c31f71be144 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:24.966251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:24.966251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:24.966251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:24.966251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:24.966251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:24.966251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:24.966251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:24.966251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:24.966251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:24.966251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:24.966251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:24.966251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:24.966251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:24.966251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:24.967073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:24.967073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:24.967073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:24.967073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:24.967073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:24.967073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:24.967073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:24.967073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:24.967073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:24.967073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:24.967073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:24.967073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:24.967073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:24.967073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:24.967073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:24.967073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:24.967073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:24.967919 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:24.967919 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:24.967919 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:24.967919 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:24.967919 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 257/506] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:24.968212 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55abcf-1dc1-42f1-bacb-9c31f71be144 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:25.072012 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55abcf-1dc1-42f1-bacb-9c31f71be144 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:25.074034 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41b778-4397-431b-b145-d8307e2989ae None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:25.107289 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41a871-82d7-4bcf-9a24-81f0fe2272d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:25.108083 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41a871-82d7-4bcf-9a24-81f0fe2272d6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:25.108386 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41a871-82d7-4bcf-9a24-81f0fe2272d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:25.108657 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41a871-82d7-4bcf-9a24-81f0fe2272d6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:25.109542 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d3730-41aa-43af-8723-ecc70deee0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:25.110229 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d3730-41aa-43af-8723-ecc70deee0fd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:25.110517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d3730-41aa-43af-8723-ecc70deee0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:25.110795 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d3730-41aa-43af-8723-ecc70deee0fd None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:25.111534 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d3730-41aa-43af-8723-ecc70deee0fd None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:25.111842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d3730-41aa-43af-8723-ecc70deee0fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:25.113225 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d3730-41aa-43af-8723-ecc70deee0fd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:25.115987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41a871-82d7-4bcf-9a24-81f0fe2272d6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:25.117427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41a871-82d7-4bcf-9a24-81f0fe2272d6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:25.121765 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 250/507] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 30 18:15:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:25.122636 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d3730-41aa-43af-8723-ecc70deee0fd None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:25.133407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-2c41a871-82d7-4bcf-9a24-81f0fe2272d6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 18:15:25.133407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:25.133407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:25.133407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:25.133407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:25.133407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:25.133407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:25.133407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:25.133407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:25.133407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:25.133407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:25.133407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:25.133407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:15:25.133407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:15:25.133407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:15:25.133407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:15:25.134431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:25.134431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:25.134431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:15:25.134431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:25.134431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:15:25.134431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:25.134431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:25.134431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:25.134431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:25.134431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:25.134431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:25.134431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:25.134431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:25.134431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:25.134431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:25.134431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:25.134431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:25.134431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:25.135299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:25.135299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:25.135299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:15:25.135299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:15:25.135299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:15:25.135299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:15:25.135299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:15:25.135299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:15:25.135299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 18:15:25.135299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:25.135299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 258/508] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:15:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:25.135839 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41a871-82d7-4bcf-9a24-81f0fe2272d6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:25.152244 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:25.181315 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:25.232940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d3730-41aa-43af-8723-ecc70deee0fd None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:25.252680 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41a871-82d7-4bcf-9a24-81f0fe2272d6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:25.271298 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e1976-f018-43f8-b7b3-d075a634397c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:25.272267 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e1976-f018-43f8-b7b3-d075a634397c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:25.272467 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e1976-f018-43f8-b7b3-d075a634397c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:25.272749 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e1976-f018-43f8-b7b3-d075a634397c None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:25.273479 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e1976-f018-43f8-b7b3-d075a634397c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:25.273828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e1976-f018-43f8-b7b3-d075a634397c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:25.275234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e1976-f018-43f8-b7b3-d075a634397c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:25.287198 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745644ac-5f7c-4031-b7cd-b2e028b13a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:25.287826 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 251/509] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:25.287950 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745644ac-5f7c-4031-b7cd-b2e028b13a12 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:25.288132 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745644ac-5f7c-4031-b7cd-b2e028b13a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:25.288418 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745644ac-5f7c-4031-b7cd-b2e028b13a12 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:25.288650 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e1976-f018-43f8-b7b3-d075a634397c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:25.289224 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745644ac-5f7c-4031-b7cd-b2e028b13a12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:25.289538 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745644ac-5f7c-4031-b7cd-b2e028b13a12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:25.290942 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745644ac-5f7c-4031-b7cd-b2e028b13a12 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:25.303698 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 259/510] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:25.304540 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745644ac-5f7c-4031-b7cd-b2e028b13a12 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:25.380370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:25.399620 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e1976-f018-43f8-b7b3-d075a634397c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:25.403870 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:25.407034 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745644ac-5f7c-4031-b7cd-b2e028b13a12 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:25.442470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0776cc-f779-4f19-b620-3e468578b1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:25.443300 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0776cc-f779-4f19-b620-3e468578b1fc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:25.444118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0776cc-f779-4f19-b620-3e468578b1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:25.444372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0776cc-f779-4f19-b620-3e468578b1fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:25.448734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3accfc-b29f-4a0f-95a0-b33d77386137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:25.449258 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3accfc-b29f-4a0f-95a0-b33d77386137 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:25.449650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3accfc-b29f-4a0f-95a0-b33d77386137 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:25.449925 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3accfc-b29f-4a0f-95a0-b33d77386137 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:25.455400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0776cc-f779-4f19-b620-3e468578b1fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:25.456945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0776cc-f779-4f19-b620-3e468578b1fc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:25.461554 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3accfc-b29f-4a0f-95a0-b33d77386137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:25.463098 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3accfc-b29f-4a0f-95a0-b33d77386137 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:25.477061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ad0776cc-f779-4f19-b620-3e468578b1fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:25.477061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:25.477061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:25.477061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:25.477061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:25.477061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:25.477061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:25.477061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:25.477061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:25.477061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:25.477061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:25.477061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:25.477061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:25.477061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:25.477061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:25.478269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:25.478269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:25.478269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:25.478269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:25.478269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:25.478269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:25.478269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:25.478269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:25.478269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:25.478269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:25.478269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:25.478269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:25.478269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:25.478269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:25.478269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:25.478269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:25.478269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:25.479827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:25.479827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:25.479827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:25.479827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:25.479827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 252/511] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:25.479827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0776cc-f779-4f19-b620-3e468578b1fc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:25.492290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ed3accfc-b29f-4a0f-95a0-b33d77386137 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:25.492290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:25.492290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:25.492290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:25.492290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:25.492290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:25.492290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:25.492290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:25.492290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:25.492290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:25.492290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:25.492290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:25.492290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:25.492290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:25.492290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:25.493550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:25.493550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:25.493550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:25.493550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:25.493550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:25.493550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:25.493550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:25.493550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:25.493550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:25.493550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:25.493550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:25.493550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:25.493550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:25.493550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:25.493550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:25.493550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:25.493550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:25.494446 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:25.494446 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:25.494446 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:25.494446 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:25.494446 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 260/512] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:25.494446 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3accfc-b29f-4a0f-95a0-b33d77386137 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:25.600439 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0776cc-f779-4f19-b620-3e468578b1fc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:25.601947 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3accfc-b29f-4a0f-95a0-b33d77386137 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:25.639811 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8c35f-3197-447f-8184-098480ec9147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:25.640480 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8c35f-3197-447f-8184-098480ec9147 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:25.640761 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8c35f-3197-447f-8184-098480ec9147 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:25.641039 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8c35f-3197-447f-8184-098480ec9147 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:25.641922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8c35f-3197-447f-8184-098480ec9147 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:25.642208 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27819f-6580-4b4b-97c6-66f76c19944d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:25.642403 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8c35f-3197-447f-8184-098480ec9147 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:25.643000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27819f-6580-4b4b-97c6-66f76c19944d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:25.643281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27819f-6580-4b4b-97c6-66f76c19944d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:25.643556 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27819f-6580-4b4b-97c6-66f76c19944d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:25.643671 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8c35f-3197-447f-8184-098480ec9147 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:25.646008 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:25.651105 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc27819f-6580-4b4b-97c6-66f76c19944d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:25.652588 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc27819f-6580-4b4b-97c6-66f76c19944d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:25.668470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-cc27819f-6580-4b4b-97c6-66f76c19944d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:25.668470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:25.668470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:25.668470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:25.668470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:25.668470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:25.668470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:25.668470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:25.668470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:25.668470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:25.668470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:25.668470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:25.668470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:15:25.668470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:15:25.668470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:15:25.669826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:15:25.669826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:25.669826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:25.669826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:25.669826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:25.669826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:25.669826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:25.669826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:25.669826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:25.669826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:25.669826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:25.669826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:25.669826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:25.669826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:25.669826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:25.669826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:25.669826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:25.670718 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:15:25.670718 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:15:25.670718 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:15:25.670718 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:15:25.670718 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:15:25.670718 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:15:25.670718 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:25.670718 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:25.670718 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 261/513] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:25.671235 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27819f-6580-4b4b-97c6-66f76c19944d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:25.689121 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 253/514] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Jun 30 18:15:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:25.690089 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8c35f-3197-447f-8184-098480ec9147 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:25.695458 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:25.788415 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27819f-6580-4b4b-97c6-66f76c19944d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:25.825809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8c35f-3197-447f-8184-098480ec9147 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:25.833018 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361a9c0-19ca-47a6-8ac9-7b287b142fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:25.834471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361a9c0-19ca-47a6-8ac9-7b287b142fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:25.834762 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361a9c0-19ca-47a6-8ac9-7b287b142fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:25.835035 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361a9c0-19ca-47a6-8ac9-7b287b142fc7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:25.843640 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0361a9c0-19ca-47a6-8ac9-7b287b142fc7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:25.845100 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0361a9c0-19ca-47a6-8ac9-7b287b142fc7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:25.860652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-0361a9c0-19ca-47a6-8ac9-7b287b142fc7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 18:15:25.860652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:25.860652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:25.860652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:25.860652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:25.860652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:25.860652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:25.860652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:25.860652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:25.860652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:25.860652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:25.860652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:25.860652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:15:25.860652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:15:25.860652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:15:25.860652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:15:25.861882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:25.861882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:25.861882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:15:25.861882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:25.861882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:15:25.861882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:25.861882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:25.861882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:25.861882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:25.861882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:25.861882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:25.861882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:25.861882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:25.861882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:25.861882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:25.861882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:25.861882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:25.861882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:25.863062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:25.863062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:25.863062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:15:25.863062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:15:25.863062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:15:25.863062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:15:25.863062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:15:25.863062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:15:25.863062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 18:15:25.863062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:25.863062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 262/515] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:15:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:25.863808 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361a9c0-19ca-47a6-8ac9-7b287b142fc7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:25.867167 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7cf1a-a101-422d-92d8-22450452cfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:25.867860 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7cf1a-a101-422d-92d8-22450452cfd0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:25.868166 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7cf1a-a101-422d-92d8-22450452cfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:25.868448 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7cf1a-a101-422d-92d8-22450452cfd0 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:25.869194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7cf1a-a101-422d-92d8-22450452cfd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:25.869610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7cf1a-a101-422d-92d8-22450452cfd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:25.870954 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7cf1a-a101-422d-92d8-22450452cfd0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:25.883320 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 254/516] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:25.884144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7cf1a-a101-422d-92d8-22450452cfd0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:25.936521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:25.938445 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:25.967140 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361a9c0-19ca-47a6-8ac9-7b287b142fc7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:25.988572 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7cf1a-a101-422d-92d8-22450452cfd0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:26.001663 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215ea43-df0b-4fdb-98c4-5820061f88cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:26.002327 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215ea43-df0b-4fdb-98c4-5820061f88cd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:26.002613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215ea43-df0b-4fdb-98c4-5820061f88cd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:26.002892 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215ea43-df0b-4fdb-98c4-5820061f88cd None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:26.003610 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3215ea43-df0b-4fdb-98c4-5820061f88cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:26.003918 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3215ea43-df0b-4fdb-98c4-5820061f88cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:26.005292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3215ea43-df0b-4fdb-98c4-5820061f88cd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:26.021160 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 263/517] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:26.021160 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215ea43-df0b-4fdb-98c4-5820061f88cd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:26.022031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7764b9-ac6c-4737-8afe-00a06ebbfcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:26.022623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7764b9-ac6c-4737-8afe-00a06ebbfcf6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:26.023134 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7764b9-ac6c-4737-8afe-00a06ebbfcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:26.024165 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7764b9-ac6c-4737-8afe-00a06ebbfcf6 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:26.024289 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7764b9-ac6c-4737-8afe-00a06ebbfcf6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:26.024578 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7764b9-ac6c-4737-8afe-00a06ebbfcf6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:26.026111 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7764b9-ac6c-4737-8afe-00a06ebbfcf6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:26.046586 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 255/518] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:15:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:26.047530 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7764b9-ac6c-4737-8afe-00a06ebbfcf6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:26.135128 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215ea43-df0b-4fdb-98c4-5820061f88cd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:26.158880 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7764b9-ac6c-4737-8afe-00a06ebbfcf6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:26.164412 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:26.170265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44df137-f716-44e6-802c-2ce655fd1852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:26.170915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44df137-f716-44e6-802c-2ce655fd1852 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:26.171202 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44df137-f716-44e6-802c-2ce655fd1852 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:26.171482 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44df137-f716-44e6-802c-2ce655fd1852 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:26.172882 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:26.201319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb303ef7-f3f9-43b4-a316-0fcca518be3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:26.202034 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb303ef7-f3f9-43b4-a316-0fcca518be3a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:26.202323 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb303ef7-f3f9-43b4-a316-0fcca518be3a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:26.202605 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb303ef7-f3f9-43b4-a316-0fcca518be3a None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:26.203449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb303ef7-f3f9-43b4-a316-0fcca518be3a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:26.203765 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb303ef7-f3f9-43b4-a316-0fcca518be3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:26.205227 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb303ef7-f3f9-43b4-a316-0fcca518be3a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:26.222400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 256/519] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Jun 30 18:15:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:26.223127 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb303ef7-f3f9-43b4-a316-0fcca518be3a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:26.243476 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44df137-f716-44e6-802c-2ce655fd1852 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:26.245455 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44df137-f716-44e6-802c-2ce655fd1852 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:26.331188 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb303ef7-f3f9-43b4-a316-0fcca518be3a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:26.343587 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 264/520] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:15:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:15:26.344428 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44df137-f716-44e6-802c-2ce655fd1852 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:26.366606 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeeb848-c24a-405d-bb2a-f822c98e78a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:26.367273 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeeb848-c24a-405d-bb2a-f822c98e78a7 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:26.367557 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeeb848-c24a-405d-bb2a-f822c98e78a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:26.367831 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeeb848-c24a-405d-bb2a-f822c98e78a7 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:26.368528 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeeb848-c24a-405d-bb2a-f822c98e78a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:26.370007 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeeb848-c24a-405d-bb2a-f822c98e78a7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:26.374776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:26.389763 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 257/521] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 30 18:15:26 2022] POST /identity/v3/users => generated 247 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:15:26.390597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeeb848-c24a-405d-bb2a-f822c98e78a7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:26.423187 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:26.477545 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44df137-f716-44e6-802c-2ce655fd1852 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:26.515408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeeb848-c24a-405d-bb2a-f822c98e78a7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:26.516596 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54a106-d0df-42a2-adff-7837957b94cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:26.517804 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54a106-d0df-42a2-adff-7837957b94cd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:26.518375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54a106-d0df-42a2-adff-7837957b94cd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:26.518659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54a106-d0df-42a2-adff-7837957b94cd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:26.528501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54a106-d0df-42a2-adff-7837957b94cd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:26.530230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54a106-d0df-42a2-adff-7837957b94cd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:26.547994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-fa54a106-d0df-42a2-adff-7837957b94cd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 18:15:26.547994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:26.547994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:26.547994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:26.547994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:26.547994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:26.547994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:26.547994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:26.547994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:26.547994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:26.547994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:26.547994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:26.547994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:15:26.547994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:15:26.547994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:15:26.547994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:15:26.548972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:26.548972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:26.548972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:15:26.548972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:26.548972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:15:26.548972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:26.548972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:26.548972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:26.548972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:26.548972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:26.548972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:26.548972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:26.548972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:26.548972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:26.548972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:26.548972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:26.548972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:26.548972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:26.549933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:26.549933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:26.549933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:15:26.549933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:15:26.549933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:15:26.549933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:15:26.549933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:15:26.549933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:15:26.549933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 18:15:26.549933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:26.549933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 265/522] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 30 18:15:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:26.550651 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54a106-d0df-42a2-adff-7837957b94cd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:26.571059 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd469ebd-cd69-40ab-9968-4a0e48822920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:26.571059 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd469ebd-cd69-40ab-9968-4a0e48822920 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:26.571518 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd469ebd-cd69-40ab-9968-4a0e48822920 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:26.573385 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd469ebd-cd69-40ab-9968-4a0e48822920 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:26.573385 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd469ebd-cd69-40ab-9968-4a0e48822920 None admin] RBAC: Adding query filter params (role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d, scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b, user.id=b8751a4e92174d18b7b138392d2579a1) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:26.573385 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd469ebd-cd69-40ab-9968-4a0e48822920 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:26.576113 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd469ebd-cd69-40ab-9968-4a0e48822920 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:26.585396 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 258/523] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:15:26 2022] GET /identity/v3/role_assignments?user.id=b8751a4e92174d18b7b138392d2579a1&scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b&role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:26.586870 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd469ebd-cd69-40ab-9968-4a0e48822920 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:26.706379 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:26.724877 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54a106-d0df-42a2-adff-7837957b94cd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:26.732539 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:26.748197 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd469ebd-cd69-40ab-9968-4a0e48822920 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:26.779988 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0913a34-0db9-42be-8d0b-a50975ef59ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:26.783068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0913a34-0db9-42be-8d0b-a50975ef59ac None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:26.784459 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0913a34-0db9-42be-8d0b-a50975ef59ac None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:26.784893 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0913a34-0db9-42be-8d0b-a50975ef59ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:26.794910 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c64b97-d9b5-43e2-b01a-2d4442bfe8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:26.795866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c64b97-d9b5-43e2-b01a-2d4442bfe8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:26.796253 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c64b97-d9b5-43e2-b01a-2d4442bfe8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:26.796634 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c64b97-d9b5-43e2-b01a-2d4442bfe8c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:26.816103 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c64b97-d9b5-43e2-b01a-2d4442bfe8c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:26.820363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c64b97-d9b5-43e2-b01a-2d4442bfe8c4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:26.867017 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-96c64b97-d9b5-43e2-b01a-2d4442bfe8c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:26.867017 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:26.867017 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:26.867017 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:26.867017 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:26.867017 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:26.867017 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:26.867017 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:26.867017 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:26.867017 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:26.867017 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:26.867017 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:26.867017 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:26.867017 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:26.867017 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:26.868130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:26.868130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:26.868130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:26.868130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:26.868130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:26.868130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:26.868130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:26.868130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:26.868130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:26.868130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:26.868130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:26.868130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:26.868130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:26.868130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:26.868130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:26.868130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:26.868130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:26.869252 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:26.869252 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:26.869252 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:26.869252 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:26.875209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 259/524] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:26.876352 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c64b97-d9b5-43e2-b01a-2d4442bfe8c4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:26.887181 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0913a34-0db9-42be-8d0b-a50975ef59ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:26.888800 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0913a34-0db9-42be-8d0b-a50975ef59ac None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:27.009769 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:27.018441 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 266/525] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:15:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:15:27.019246 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0913a34-0db9-42be-8d0b-a50975ef59ac None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:27.047209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c64b97-d9b5-43e2-b01a-2d4442bfe8c4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:27.087527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d0405-734b-43c1-af3d-b54db69e1fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:27.088830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d0405-734b-43c1-af3d-b54db69e1fdb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:27.089476 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d0405-734b-43c1-af3d-b54db69e1fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:27.090142 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d0405-734b-43c1-af3d-b54db69e1fdb None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:27.091312 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d0405-734b-43c1-af3d-b54db69e1fdb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:27.091862 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d0405-734b-43c1-af3d-b54db69e1fdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:27.093558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d0405-734b-43c1-af3d-b54db69e1fdb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:27.114543 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:27.120576 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 260/526] 10.222.0.33 () {56 vars in 1139 bytes} [Thu Jun 30 18:15:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:27.121591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d0405-734b-43c1-af3d-b54db69e1fdb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:27.160253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0913a34-0db9-42be-8d0b-a50975ef59ac None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:27.198832 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130f0fd-f775-455d-8b76-0257b28e47fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:27.199960 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130f0fd-f775-455d-8b76-0257b28e47fd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:27.200579 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130f0fd-f775-455d-8b76-0257b28e47fd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:27.201191 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130f0fd-f775-455d-8b76-0257b28e47fd None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:27.202395 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130f0fd-f775-455d-8b76-0257b28e47fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:27.203055 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130f0fd-f775-455d-8b76-0257b28e47fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:27.204657 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130f0fd-f775-455d-8b76-0257b28e47fd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:27.219531 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 267/527] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:27.220815 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130f0fd-f775-455d-8b76-0257b28e47fd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:27.236233 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d0405-734b-43c1-af3d-b54db69e1fdb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:27.277805 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f282e-ddb9-4400-8233-0a8eeed2df32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:27.278521 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f282e-ddb9-4400-8233-0a8eeed2df32 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:27.278843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f282e-ddb9-4400-8233-0a8eeed2df32 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:27.279075 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f282e-ddb9-4400-8233-0a8eeed2df32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:27.284653 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:27.292146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f282e-ddb9-4400-8233-0a8eeed2df32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:27.295147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f282e-ddb9-4400-8233-0a8eeed2df32 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:27.317290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-271f282e-ddb9-4400-8233-0a8eeed2df32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:27.317290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:27.317290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:27.317290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:27.317290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:27.317290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:27.317290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:27.317290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:27.317290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:27.317290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:27.317290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:27.317290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:27.317290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:27.317290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:27.317290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:27.318381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:27.318381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:27.318381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:27.318381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:27.318381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:27.318381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:27.318381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:27.318381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:27.318381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:27.318381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:27.318381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:27.318381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:27.318381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:27.318381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:27.318381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:27.318381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:27.318381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:27.319324 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:27.319324 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:27.319324 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:27.319324 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:27.321148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 261/528] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:27.322256 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f282e-ddb9-4400-8233-0a8eeed2df32 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:27.341085 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130f0fd-f775-455d-8b76-0257b28e47fd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:27.378697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8d5e5-34d6-4fbe-976e-1c189b02398b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:27.379501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8d5e5-34d6-4fbe-976e-1c189b02398b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:27.379890 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8d5e5-34d6-4fbe-976e-1c189b02398b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:27.380293 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8d5e5-34d6-4fbe-976e-1c189b02398b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:27.381979 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:27.435746 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f282e-ddb9-4400-8233-0a8eeed2df32 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:27.472649 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9c421-38d5-45c7-bd9d-602309ce5296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:27.473475 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9c421-38d5-45c7-bd9d-602309ce5296 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:27.473807 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9c421-38d5-45c7-bd9d-602309ce5296 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:27.474085 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9c421-38d5-45c7-bd9d-602309ce5296 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:27.481772 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de9c421-38d5-45c7-bd9d-602309ce5296 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:27.483233 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de9c421-38d5-45c7-bd9d-602309ce5296 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:27.489414 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8d5e5-34d6-4fbe-976e-1c189b02398b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:27.492421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8d5e5-34d6-4fbe-976e-1c189b02398b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:27.498822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-9de9c421-38d5-45c7-bd9d-602309ce5296 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:27.498822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:27.498822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:27.498822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:27.498822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:27.498822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:27.498822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:27.498822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:27.498822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:27.498822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:27.498822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:27.498822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:27.498822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:15:27.498822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:15:27.498822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:15:27.499658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:15:27.499658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:27.499658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:27.499658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:27.499658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:27.499658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:27.499658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:27.499658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:27.499658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:27.499658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:27.499658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:27.499658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:27.499658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:27.499658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:27.499658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:27.499658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:27.499658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:27.500502 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:15:27.500502 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:15:27.500502 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:15:27.500502 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:15:27.500502 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:15:27.500502 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:15:27.500502 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:27.500502 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:27.500502 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 262/529] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:27.501401 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9c421-38d5-45c7-bd9d-602309ce5296 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:27.564727 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:27.623308 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9c421-38d5-45c7-bd9d-602309ce5296 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:27.639251 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:27.640203 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 268/530] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:15:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:15:27.641064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8d5e5-34d6-4fbe-976e-1c189b02398b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:27.685019 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98c86a-4e64-4b80-b41f-a0a229f28baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:27.685738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98c86a-4e64-4b80-b41f-a0a229f28baa None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:27.686029 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98c86a-4e64-4b80-b41f-a0a229f28baa None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:27.686309 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98c86a-4e64-4b80-b41f-a0a229f28baa None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:27.687406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98c86a-4e64-4b80-b41f-a0a229f28baa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:27.687878 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98c86a-4e64-4b80-b41f-a0a229f28baa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:27.690472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98c86a-4e64-4b80-b41f-a0a229f28baa None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:27.709820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 263/531] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:27.711837 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98c86a-4e64-4b80-b41f-a0a229f28baa None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:27.768749 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8d5e5-34d6-4fbe-976e-1c189b02398b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:27.821475 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd34b2c-79b3-4223-b502-8e6f6654a2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:27.822102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd34b2c-79b3-4223-b502-8e6f6654a2f5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:27.822388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd34b2c-79b3-4223-b502-8e6f6654a2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:27.822669 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd34b2c-79b3-4223-b502-8e6f6654a2f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:27.823473 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd34b2c-79b3-4223-b502-8e6f6654a2f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:27.823789 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd34b2c-79b3-4223-b502-8e6f6654a2f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:27.825270 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd34b2c-79b3-4223-b502-8e6f6654a2f5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:27.846909 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 269/532] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:15:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:27.848155 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd34b2c-79b3-4223-b502-8e6f6654a2f5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:27.860012 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98c86a-4e64-4b80-b41f-a0a229f28baa None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:27.863938 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:27.906524 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973f224-249d-4975-86e9-3663be0907ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:27.907323 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973f224-249d-4975-86e9-3663be0907ee None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:27.907608 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973f224-249d-4975-86e9-3663be0907ee None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:27.907889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973f224-249d-4975-86e9-3663be0907ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:27.922437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973f224-249d-4975-86e9-3663be0907ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:27.924179 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973f224-249d-4975-86e9-3663be0907ee None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:27.947958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-2973f224-249d-4975-86e9-3663be0907ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:27.947958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:27.947958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:27.947958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:27.947958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:27.947958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:27.947958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:27.947958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:27.947958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:27.947958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:27.947958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:27.947958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:27.947958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:27.947958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:27.947958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:27.949027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:27.949027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:27.949027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:27.949027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:27.949027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:27.949027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:27.949027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:27.949027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:27.949027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:27.949027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:27.949027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:27.949027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:27.949027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:27.949027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:27.949027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:27.949027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:27.949027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:27.950027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:27.950027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:27.950027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:27.950027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:27.953788 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 264/533] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:27.954761 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973f224-249d-4975-86e9-3663be0907ee None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:27.962804 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd34b2c-79b3-4223-b502-8e6f6654a2f5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:27.987707 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:28.002376 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593efb1b-b1c6-4a0e-af64-a8ea50754f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:28.003415 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593efb1b-b1c6-4a0e-af64-a8ea50754f20 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:28.003877 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593efb1b-b1c6-4a0e-af64-a8ea50754f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:28.004331 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593efb1b-b1c6-4a0e-af64-a8ea50754f20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:28.012988 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593efb1b-b1c6-4a0e-af64-a8ea50754f20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:28.015019 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593efb1b-b1c6-4a0e-af64-a8ea50754f20 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:28.033505 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-593efb1b-b1c6-4a0e-af64-a8ea50754f20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:28.033505 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:28.033505 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:28.033505 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:28.033505 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:28.033505 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:28.033505 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:28.033505 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:28.033505 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:28.033505 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:28.033505 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:28.033505 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:28.033505 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:15:28.033505 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:15:28.033505 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:15:28.034371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:15:28.034371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:28.034371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:28.034371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:28.034371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:28.034371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:28.034371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:28.034371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:28.034371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:28.034371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:28.034371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:28.034371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:28.034371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:28.034371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:28.034371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:28.034371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:28.034371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:28.035193 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:15:28.035193 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:15:28.035193 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:15:28.035193 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:15:28.035193 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:15:28.035193 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:15:28.035193 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:28.035193 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:28.038306 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 270/534] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:28.039394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593efb1b-b1c6-4a0e-af64-a8ea50754f20 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:28.085547 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973f224-249d-4975-86e9-3663be0907ee None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:28.129358 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:28.137369 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965743f-c586-4507-8204-18fe562e0c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:28.138058 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965743f-c586-4507-8204-18fe562e0c6c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:28.138338 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965743f-c586-4507-8204-18fe562e0c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:28.138611 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965743f-c586-4507-8204-18fe562e0c6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:28.139447 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965743f-c586-4507-8204-18fe562e0c6c None admin] RBAC: Adding query filter params (role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d, scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b, user.id=37a9fea702cd405ea9fc14ec2c689648) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:28.139758 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965743f-c586-4507-8204-18fe562e0c6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:28.141904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965743f-c586-4507-8204-18fe562e0c6c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:28.152338 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 265/535] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:15:27 2022] GET /identity/v3/role_assignments?user.id=37a9fea702cd405ea9fc14ec2c689648&scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b&role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:28.161665 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965743f-c586-4507-8204-18fe562e0c6c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:28.198090 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593efb1b-b1c6-4a0e-af64-a8ea50754f20 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:28.277971 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44aeb22-0e9c-4e11-936c-f6a92a0af3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:28.279162 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44aeb22-0e9c-4e11-936c-f6a92a0af3cf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:28.279808 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44aeb22-0e9c-4e11-936c-f6a92a0af3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:28.280265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44aeb22-0e9c-4e11-936c-f6a92a0af3cf None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:28.281203 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44aeb22-0e9c-4e11-936c-f6a92a0af3cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:28.281701 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44aeb22-0e9c-4e11-936c-f6a92a0af3cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:28.283323 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44aeb22-0e9c-4e11-936c-f6a92a0af3cf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:28.311340 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 271/536] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:28.316048 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44aeb22-0e9c-4e11-936c-f6a92a0af3cf None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:28.377404 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:28.402285 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965743f-c586-4507-8204-18fe562e0c6c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:28.444469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b876b-ec70-47bf-bc2c-434156093566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:28.445378 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b876b-ec70-47bf-bc2c-434156093566 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:28.445866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b876b-ec70-47bf-bc2c-434156093566 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:28.446396 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b876b-ec70-47bf-bc2c-434156093566 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:28.447344 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665b876b-ec70-47bf-bc2c-434156093566 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:28.447805 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665b876b-ec70-47bf-bc2c-434156093566 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:28.449667 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665b876b-ec70-47bf-bc2c-434156093566 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:28.476555 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 266/537] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:15:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:28.477683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44aeb22-0e9c-4e11-936c-f6a92a0af3cf None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:28.478032 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b876b-ec70-47bf-bc2c-434156093566 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:28.513382 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:28.519598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab002a0-d84a-4f82-9a98-faf8a927feff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:28.520403 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab002a0-d84a-4f82-9a98-faf8a927feff None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:28.520783 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab002a0-d84a-4f82-9a98-faf8a927feff None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:28.521162 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab002a0-d84a-4f82-9a98-faf8a927feff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:28.530666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab002a0-d84a-4f82-9a98-faf8a927feff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:28.532400 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab002a0-d84a-4f82-9a98-faf8a927feff None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:28.549528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-fab002a0-d84a-4f82-9a98-faf8a927feff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:28.549528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:28.549528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:28.549528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:28.549528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:28.549528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:28.549528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:28.549528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:28.549528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:28.549528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:28.549528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:28.549528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:28.549528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:15:28.549528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:15:28.549528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:15:28.550697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:15:28.550697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:28.550697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:28.550697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:28.550697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:28.550697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:28.550697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:28.550697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:28.550697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:28.550697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:28.550697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:28.550697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:28.550697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:28.550697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:28.550697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:28.550697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:28.550697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:28.551792 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:15:28.551792 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:15:28.551792 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:15:28.551792 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:15:28.551792 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:15:28.551792 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:15:28.551792 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:28.551792 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:28.559041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 272/538] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:28.567035 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab002a0-d84a-4f82-9a98-faf8a927feff None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:28.630321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b876b-ec70-47bf-bc2c-434156093566 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:28.686247 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e801c3d-4f56-44e8-9005-af110cf1a055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:28.687677 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e801c3d-4f56-44e8-9005-af110cf1a055 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:28.688388 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e801c3d-4f56-44e8-9005-af110cf1a055 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:28.689080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e801c3d-4f56-44e8-9005-af110cf1a055 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:28.690953 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e801c3d-4f56-44e8-9005-af110cf1a055 None admin] RBAC: Adding query filter params (role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d, scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b, user.id=92c70ce830ea4dd981716d622b16d595) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:28.691719 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e801c3d-4f56-44e8-9005-af110cf1a055 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:28.695276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e801c3d-4f56-44e8-9005-af110cf1a055 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:28.706577 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 267/539] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:15:28 2022] GET /identity/v3/role_assignments?user.id=92c70ce830ea4dd981716d622b16d595&scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b&role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:28.765151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab002a0-d84a-4f82-9a98-faf8a927feff None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:28.789188 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0a818-d671-44a1-aa86-26fd42f94b28 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:28.790209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0a818-d671-44a1-aa86-26fd42f94b28 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:28.790620 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0a818-d671-44a1-aa86-26fd42f94b28 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:28.797333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 268/540] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:15:28.799462 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199567e-d7eb-4918-bc13-255f8b6917bc None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:28.799752 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199567e-d7eb-4918-bc13-255f8b6917bc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:28.800036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199567e-d7eb-4918-bc13-255f8b6917bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:28.820948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5199567e-d7eb-4918-bc13-255f8b6917bc None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:28.822561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5199567e-d7eb-4918-bc13-255f8b6917bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:28.838029 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:28.844919 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1e691-ff41-4868-9008-f7f3cc0a70b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:28.845642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1e691-ff41-4868-9008-f7f3cc0a70b6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:28.845941 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1e691-ff41-4868-9008-f7f3cc0a70b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:28.846227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1e691-ff41-4868-9008-f7f3cc0a70b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:28.847148 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc1e691-ff41-4868-9008-f7f3cc0a70b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:28.847470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc1e691-ff41-4868-9008-f7f3cc0a70b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:28.848987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc1e691-ff41-4868-9008-f7f3cc0a70b6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:28.864393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 273/541] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:15:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:28.874926 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1e691-ff41-4868-9008-f7f3cc0a70b6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:28.942138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:28.979194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 269/542] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 18:15:28.982025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfcd08-cbd7-4d65-9312-cc468cec1fc6 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:28.982840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfcd08-cbd7-4d65-9312-cc468cec1fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:28.983615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfcd08-cbd7-4d65-9312-cc468cec1fc6 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:28.993418 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 270/543] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:28.996838 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef697e-326c-46ba-a559-be36ddde43d8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:28.997694 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef697e-326c-46ba-a559-be36ddde43d8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:28.998482 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef697e-326c-46ba-a559-be36ddde43d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:29.013180 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1e691-ff41-4868-9008-f7f3cc0a70b6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:29.029912 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-dbef697e-326c-46ba-a559-be36ddde43d8 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:29.032680 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dbef697e-326c-46ba-a559-be36ddde43d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:29.071166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd566e9-c491-449c-9b7e-cf749e665654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:29.071380 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd566e9-c491-449c-9b7e-cf749e665654 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:29.071643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd566e9-c491-449c-9b7e-cf749e665654 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:29.071935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd566e9-c491-449c-9b7e-cf749e665654 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:29.072832 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd566e9-c491-449c-9b7e-cf749e665654 None admin] RBAC: Adding query filter params (role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d, scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b, user.id=1d8c54102fb340859ed0acc3189fe376) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:29.073164 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd566e9-c491-449c-9b7e-cf749e665654 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:29.074731 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd566e9-c491-449c-9b7e-cf749e665654 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:29.081529 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 274/544] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:15:28 2022] GET /identity/v3/role_assignments?user.id=1d8c54102fb340859ed0acc3189fe376&scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b&role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:29.082552 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd566e9-c491-449c-9b7e-cf749e665654 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:29.129975 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 271/545] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 18:15:29.156173 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef697e-326c-46ba-a559-be36ddde43d8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:29.206943 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd566e9-c491-449c-9b7e-cf749e665654 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:29.238053 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:29.256366 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:29.265307 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a161cec9-5edb-4686-9521-30e4e1cb0961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:29.266183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a161cec9-5edb-4686-9521-30e4e1cb0961 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:29.266474 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a161cec9-5edb-4686-9521-30e4e1cb0961 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:29.266759 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a161cec9-5edb-4686-9521-30e4e1cb0961 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:29.280390 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a161cec9-5edb-4686-9521-30e4e1cb0961 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:29.289429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a161cec9-5edb-4686-9521-30e4e1cb0961 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:29.289429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:29.289429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:29.289429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:29.289429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:29.289429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:29.289429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:29.289429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:29.289429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:29.289429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:29.289429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:29.289429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:29.289429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:29.289429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:29.289429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:29.289429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:29.290468 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:29.290468 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:29.290468 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:29.290468 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:29.290468 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:29.290468 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:29.290468 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:29.290468 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:29.290468 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:29.290468 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:29.290468 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:29.290468 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:29.290468 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:29.290468 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:29.290468 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:29.290468 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:29.291255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:29.291255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:29.291255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:29.291255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:29.291255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:29.291255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:29.291255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:29.291255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:29.291255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:29.291255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:29.291255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:29.291255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:29.291863 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 275/546] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 30 18:15:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:29.297856 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a161cec9-5edb-4686-9521-30e4e1cb0961 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:29.359225 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef697e-326c-46ba-a559-be36ddde43d8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:29.439075 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed74fb1-ff15-433a-a4c2-42a392b4c556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:29.440661 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed74fb1-ff15-433a-a4c2-42a392b4c556 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:29.442582 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed74fb1-ff15-433a-a4c2-42a392b4c556 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:29.443082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed74fb1-ff15-433a-a4c2-42a392b4c556 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:29.467839 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed74fb1-ff15-433a-a4c2-42a392b4c556 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:29.476875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3ed74fb1-ff15-433a-a4c2-42a392b4c556 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:29.476875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:29.476875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:29.476875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:29.476875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:29.476875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:29.476875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:29.476875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:29.476875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:29.476875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:29.476875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:29.476875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:29.476875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:29.476875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:29.476875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:29.476875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:29.477828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:29.477828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:29.477828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:29.477828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:29.477828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:29.477828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:29.477828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:29.477828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:29.477828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:29.477828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:29.477828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:29.477828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:29.477828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:29.477828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:29.477828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:29.477828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:29.478614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:29.478614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:29.478614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:29.478614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:29.478614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:29.478614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:29.478614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:29.478614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:29.478614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:29.478614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:29.478614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:29.478614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:29.488504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 272/547] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 30 18:15:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:15:29.496075 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed74fb1-ff15-433a-a4c2-42a392b4c556 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:29.503933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a161cec9-5edb-4686-9521-30e4e1cb0961 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:29.594348 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948c78f-d02b-4219-94e6-929272a77f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:29.595136 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948c78f-d02b-4219-94e6-929272a77f5e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:29.595358 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948c78f-d02b-4219-94e6-929272a77f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:29.595644 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948c78f-d02b-4219-94e6-929272a77f5e None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:29.596422 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6948c78f-d02b-4219-94e6-929272a77f5e None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:29.596770 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6948c78f-d02b-4219-94e6-929272a77f5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:29.598191 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6948c78f-d02b-4219-94e6-929272a77f5e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:29.626243 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 276/548] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 30 18:15:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:29.632605 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948c78f-d02b-4219-94e6-929272a77f5e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:29.667739 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed74fb1-ff15-433a-a4c2-42a392b4c556 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:29.704132 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915eb61-0df9-4eea-912e-c74b9d16af37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:29.704813 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915eb61-0df9-4eea-912e-c74b9d16af37 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:29.705102 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915eb61-0df9-4eea-912e-c74b9d16af37 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:29.705390 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915eb61-0df9-4eea-912e-c74b9d16af37 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:29.706179 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915eb61-0df9-4eea-912e-c74b9d16af37 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:29.706492 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915eb61-0df9-4eea-912e-c74b9d16af37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:29.707889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915eb61-0df9-4eea-912e-c74b9d16af37 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:29.718159 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 273/549] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 30 18:15:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:29.718798 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:29.723718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915eb61-0df9-4eea-912e-c74b9d16af37 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:29.762019 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:29.890579 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915eb61-0df9-4eea-912e-c74b9d16af37 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:29.947428 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694bfda-d721-4736-a89b-2bcd3630f15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:29.948879 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694bfda-d721-4736-a89b-2bcd3630f15b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:29.949402 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694bfda-d721-4736-a89b-2bcd3630f15b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:29.949912 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694bfda-d721-4736-a89b-2bcd3630f15b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:29.964534 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b694bfda-d721-4736-a89b-2bcd3630f15b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:29.966632 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b694bfda-d721-4736-a89b-2bcd3630f15b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:29.967994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948c78f-d02b-4219-94e6-929272a77f5e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:29.989693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b694bfda-d721-4736-a89b-2bcd3630f15b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:29.989693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:29.989693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:29.989693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:29.989693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:29.989693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:29.989693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:29.989693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:29.989693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:29.989693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:29.989693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:29.989693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:29.989693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:29.989693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:29.989693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:29.991666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:29.991666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:29.991666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:29.991666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:29.991666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:29.991666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:29.991666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:29.991666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:29.991666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:29.991666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:29.991666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:29.991666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:29.991666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:29.991666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:29.991666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:29.991666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:29.991666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:29.992618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:29.992618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:29.992618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:29.992618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:29.995697 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 274/550] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:30.001678 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694bfda-d721-4736-a89b-2bcd3630f15b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:30.065720 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:30.070457 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c5217-265b-4f24-a115-9a82694092f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:30.071375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c5217-265b-4f24-a115-9a82694092f6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:30.071651 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c5217-265b-4f24-a115-9a82694092f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:30.071968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c5217-265b-4f24-a115-9a82694092f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:30.094219 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c5217-265b-4f24-a115-9a82694092f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:30.096360 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c5217-265b-4f24-a115-9a82694092f6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:30.129892 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694bfda-d721-4736-a89b-2bcd3630f15b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:30.138702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d97c5217-265b-4f24-a115-9a82694092f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:30.138702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:30.138702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:30.138702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:30.138702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:30.138702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:30.138702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:30.138702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:30.138702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:30.138702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:30.138702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:30.138702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:30.138702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:30.138702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:30.138702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:30.139635 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:30.139635 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:30.139635 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:30.139635 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:30.139635 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:30.139635 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:30.139635 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:30.139635 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:30.139635 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:30.139635 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:30.139635 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:30.139635 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:30.139635 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:30.139635 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:30.139635 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:30.139635 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:30.139635 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:30.140668 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:30.140668 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:30.140668 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:30.140668 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:30.140668 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 277/551] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 508 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:15:30.147341 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c5217-265b-4f24-a115-9a82694092f6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:30.171430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7de0e1-070e-4a31-a233-5c580fafad5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:30.172582 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7de0e1-070e-4a31-a233-5c580fafad5a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:30.173169 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7de0e1-070e-4a31-a233-5c580fafad5a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:30.173690 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7de0e1-070e-4a31-a233-5c580fafad5a None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:30.174798 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7de0e1-070e-4a31-a233-5c580fafad5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:30.175311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7de0e1-070e-4a31-a233-5c580fafad5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:30.176915 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7de0e1-070e-4a31-a233-5c580fafad5a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:30.190718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 275/552] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:30.201333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7de0e1-070e-4a31-a233-5c580fafad5a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:30.280058 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:30.291614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c5217-265b-4f24-a115-9a82694092f6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:30.338814 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0273280-06b2-4e80-b9e7-e63706539dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:30.339660 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0273280-06b2-4e80-b9e7-e63706539dbf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:30.340078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0273280-06b2-4e80-b9e7-e63706539dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:30.340483 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0273280-06b2-4e80-b9e7-e63706539dbf None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:30.341343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0273280-06b2-4e80-b9e7-e63706539dbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:30.341677 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0273280-06b2-4e80-b9e7-e63706539dbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:30.344615 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0273280-06b2-4e80-b9e7-e63706539dbf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:30.370171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 278/553] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:30.371500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0273280-06b2-4e80-b9e7-e63706539dbf None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:30.441090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7de0e1-070e-4a31-a233-5c580fafad5a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:30.465958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:30.481704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0273280-06b2-4e80-b9e7-e63706539dbf None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:30.529321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6aab0-d02d-43ec-89ca-b18686f320a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:30.531115 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6aab0-d02d-43ec-89ca-b18686f320a9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:30.531340 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6aab0-d02d-43ec-89ca-b18686f320a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:30.531622 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6aab0-d02d-43ec-89ca-b18686f320a9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:30.543175 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6aab0-d02d-43ec-89ca-b18686f320a9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:30.545072 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6aab0-d02d-43ec-89ca-b18686f320a9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:30.559824 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf703dee-98c1-42c6-8c88-caeb16b3c730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:30.560984 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf703dee-98c1-42c6-8c88-caeb16b3c730 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:30.561467 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf703dee-98c1-42c6-8c88-caeb16b3c730 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:30.561999 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf703dee-98c1-42c6-8c88-caeb16b3c730 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:30.578830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf703dee-98c1-42c6-8c88-caeb16b3c730 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:30.580656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf703dee-98c1-42c6-8c88-caeb16b3c730 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:30.581710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a5e6aab0-d02d-43ec-89ca-b18686f320a9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 18:15:30.581710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:30.581710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:30.581710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:30.581710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:30.581710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:30.581710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:30.581710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:30.581710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:30.581710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:30.581710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:30.581710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:30.581710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:15:30.581710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:15:30.581710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:15:30.581710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:15:30.582626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:30.582626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:30.582626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:15:30.582626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:30.582626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:15:30.582626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:30.582626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:30.582626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:30.582626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:30.582626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:30.582626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:30.582626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:30.582626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:30.582626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:30.582626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:30.582626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:30.582626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:30.582626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:30.583526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:30.583526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:30.583526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:15:30.583526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:15:30.583526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:15:30.583526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:15:30.583526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:15:30.583526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:15:30.583526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 18:15:30.583526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:30.588915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 279/554] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:15:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:30.593409 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6aab0-d02d-43ec-89ca-b18686f320a9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:30.612857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-cf703dee-98c1-42c6-8c88-caeb16b3c730 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 18:15:30.612857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:30.612857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:30.612857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:30.612857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:30.612857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:30.612857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:30.612857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:30.612857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:30.612857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:30.612857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:30.612857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:30.612857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:15:30.612857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:15:30.612857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:15:30.612857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:15:30.614014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:30.614014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:30.614014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:15:30.614014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:30.614014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:15:30.614014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:30.614014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:30.614014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:30.614014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:30.614014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:30.614014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:30.614014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:30.614014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:30.614014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:30.614014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:30.614014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:30.614014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:30.614014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:30.615957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:30.615957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:30.615957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:15:30.615957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:15:30.615957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:15:30.615957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:15:30.615957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:15:30.615957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:15:30.615957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 18:15:30.615957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:30.623344 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 276/555] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 30 18:15:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:30.627251 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf703dee-98c1-42c6-8c88-caeb16b3c730 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:30.629952 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:30.809274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6aab0-d02d-43ec-89ca-b18686f320a9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:30.837733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf703dee-98c1-42c6-8c88-caeb16b3c730 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:30.881719 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3fd54-961c-4688-aa99-bdecafb5f235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:30.882452 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3fd54-961c-4688-aa99-bdecafb5f235 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:30.882740 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3fd54-961c-4688-aa99-bdecafb5f235 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:30.883568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3fd54-961c-4688-aa99-bdecafb5f235 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:30.888044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3fd54-961c-4688-aa99-bdecafb5f235 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:30.888044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3fd54-961c-4688-aa99-bdecafb5f235 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:30.894339 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3fd54-961c-4688-aa99-bdecafb5f235 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:30.911353 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a045ff-78b8-4a3f-b21a-ee0614f54233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:30.912187 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a045ff-78b8-4a3f-b21a-ee0614f54233 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:30.912470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a045ff-78b8-4a3f-b21a-ee0614f54233 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:30.912747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a045ff-78b8-4a3f-b21a-ee0614f54233 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:30.913627 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a045ff-78b8-4a3f-b21a-ee0614f54233 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:30.913944 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a045ff-78b8-4a3f-b21a-ee0614f54233 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:30.915419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a045ff-78b8-4a3f-b21a-ee0614f54233 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:30.921846 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 280/556] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Jun 30 18:15:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:30.923512 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9edd65-e666-41b2-b278-b18e629fd57f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:30.923808 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9edd65-e666-41b2-b278-b18e629fd57f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:30.924092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9edd65-e666-41b2-b278-b18e629fd57f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:30.925531 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 281/557] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:30.926993 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9edd65-e666-41b2-b278-b18e629fd57f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:30.940190 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 277/558] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 30 18:15:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:30.941747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ade4d-cf87-45d7-8d50-ad61786019e3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:30.942046 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ade4d-cf87-45d7-8d50-ad61786019e3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:30.942366 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ade4d-cf87-45d7-8d50-ad61786019e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:30.963432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-cf5ade4d-cf87-45d7-8d50-ad61786019e3 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:30.964676 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cf5ade4d-cf87-45d7-8d50-ad61786019e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:30.986839 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:31.035512 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:31.057695 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 278/559] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:31.059584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ade4d-cf87-45d7-8d50-ad61786019e3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:31.081413 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9edd65-e666-41b2-b278-b18e629fd57f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:31.128033 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0801c3-d915-408f-a40d-450f54245208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:31.128757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0801c3-d915-408f-a40d-450f54245208 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:31.129082 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0801c3-d915-408f-a40d-450f54245208 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:31.129413 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0801c3-d915-408f-a40d-450f54245208 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:31.222155 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0801c3-d915-408f-a40d-450f54245208 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:31.224351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0801c3-d915-408f-a40d-450f54245208 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:31.226555 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ade4d-cf87-45d7-8d50-ad61786019e3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:31.280344 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de4022-6345-41f1-a828-45126891f440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:31.281143 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de4022-6345-41f1-a828-45126891f440 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:31.281425 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de4022-6345-41f1-a828-45126891f440 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:31.281744 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de4022-6345-41f1-a828-45126891f440 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:31.295260 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7de4022-6345-41f1-a828-45126891f440 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:31.296759 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7de4022-6345-41f1-a828-45126891f440 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:31.316580 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b7de4022-6345-41f1-a828-45126891f440 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:31.316580 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:31.316580 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:31.316580 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:31.316580 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:31.316580 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:31.316580 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:31.316580 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:31.316580 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:31.316580 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:31.316580 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:31.316580 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:31.316580 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:31.316580 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:31.316580 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:31.317570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:31.317570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:31.317570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:31.317570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:31.317570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:31.317570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:31.317570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:31.317570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:31.317570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:31.317570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:31.317570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:31.317570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:31.317570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:31.317570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:31.317570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:31.317570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:31.317570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:31.319088 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:31.319088 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:31.319088 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:31.319088 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:31.323078 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 279/560] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:31.323913 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de4022-6345-41f1-a828-45126891f440 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:31.334588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:31.337727 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:31.388363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 282/561] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:15:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 18:15:31.391041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad7554-5161-4071-b635-008fb11d0b14 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:31.391605 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad7554-5161-4071-b635-008fb11d0b14 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:31.392080 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad7554-5161-4071-b635-008fb11d0b14 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:31.394718 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 283/562] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:31.396782 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad7554-5161-4071-b635-008fb11d0b14 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:31.491799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de4022-6345-41f1-a828-45126891f440 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:31.542886 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ebebf9-2f39-4f36-83b7-6ab890dd5a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:31.543693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebebf9-2f39-4f36-83b7-6ab890dd5a8e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:31.543983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebebf9-2f39-4f36-83b7-6ab890dd5a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:31.544512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebebf9-2f39-4f36-83b7-6ab890dd5a8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:31.556642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ebebf9-2f39-4f36-83b7-6ab890dd5a8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:31.559417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad7554-5161-4071-b635-008fb11d0b14 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:31.564552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-89ebebf9-2f39-4f36-83b7-6ab890dd5a8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:31.564552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:31.564552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:31.564552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:31.564552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:31.564552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:31.564552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:31.564552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:31.564552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:31.564552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:31.564552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:31.564552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:31.564552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:31.564552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:31.564552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:31.564552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:31.565504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:31.565504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:31.565504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:31.565504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:31.565504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:31.565504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:31.565504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:31.565504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:31.565504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:31.565504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:31.565504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:31.565504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:31.565504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:31.565504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:31.565504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:31.565504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:31.566449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:31.566449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:31.566449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:31.566449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:31.566449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:31.566449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:31.566449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:31.566449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:31.566449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:31.566449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:31.566449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:31.566449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:31.571481 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 280/563] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 30 18:15:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:31.572199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ebebf9-2f39-4f36-83b7-6ab890dd5a8e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:31.601956 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebe3d3-65a4-43cc-ba28-c02e9e14aa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:31.603136 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebe3d3-65a4-43cc-ba28-c02e9e14aa3f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:31.603480 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebe3d3-65a4-43cc-ba28-c02e9e14aa3f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:31.603720 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebe3d3-65a4-43cc-ba28-c02e9e14aa3f None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:31.604452 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ebe3d3-65a4-43cc-ba28-c02e9e14aa3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:31.604754 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ebe3d3-65a4-43cc-ba28-c02e9e14aa3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:31.606339 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ebe3d3-65a4-43cc-ba28-c02e9e14aa3f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:31.625917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 284/564] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:31.625917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85529e-a5db-45e1-9aaa-80a4bffa907f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:31.625917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85529e-a5db-45e1-9aaa-80a4bffa907f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:31.625917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85529e-a5db-45e1-9aaa-80a4bffa907f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:31.647160 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4a85529e-a5db-45e1-9aaa-80a4bffa907f None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:31.648599 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4a85529e-a5db-45e1-9aaa-80a4bffa907f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:31.660179 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:31.681875 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:31.712078 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ebebf9-2f39-4f36-83b7-6ab890dd5a8e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:31.761624 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 285/565] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:31.761624 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85529e-a5db-45e1-9aaa-80a4bffa907f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:31.773096 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345dc48c-b215-479d-a4cc-778a9b963c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:31.774040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345dc48c-b215-479d-a4cc-778a9b963c41 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:31.774397 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345dc48c-b215-479d-a4cc-778a9b963c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:31.774751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345dc48c-b215-479d-a4cc-778a9b963c41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:31.789475 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345dc48c-b215-479d-a4cc-778a9b963c41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:31.791311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345dc48c-b215-479d-a4cc-778a9b963c41 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:31.810164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-345dc48c-b215-479d-a4cc-778a9b963c41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:31.810164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:31.810164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:31.810164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:31.810164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:31.810164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:31.810164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:31.810164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:31.810164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:31.810164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:31.810164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:31.810164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:31.810164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:31.810164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:31.810164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:31.811147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:31.811147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:31.811147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:31.811147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:31.811147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:31.811147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:31.811147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:31.811147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:31.811147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:31.811147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:31.811147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:31.811147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:31.811147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:31.811147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:31.811147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:31.811147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:31.811147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:31.812044 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:31.812044 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:31.812044 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:31.812044 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:31.812044 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 281/566] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:31.812325 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345dc48c-b215-479d-a4cc-778a9b963c41 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:31.891457 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85529e-a5db-45e1-9aaa-80a4bffa907f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:31.926162 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345dc48c-b215-479d-a4cc-778a9b963c41 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:31.927839 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2135f-05da-411d-8b2b-f57174cc3e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:31.928715 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2135f-05da-411d-8b2b-f57174cc3e17 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:31.928880 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2135f-05da-411d-8b2b-f57174cc3e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:31.929175 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2135f-05da-411d-8b2b-f57174cc3e17 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:31.929969 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2135f-05da-411d-8b2b-f57174cc3e17 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:31.930277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2135f-05da-411d-8b2b-f57174cc3e17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:31.931975 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2135f-05da-411d-8b2b-f57174cc3e17 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:31.941254 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 286/567] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 30 18:15:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:31.943570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11d3d8-c575-4652-8b74-00ef1499c0f6 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:31.943570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11d3d8-c575-4652-8b74-00ef1499c0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:31.943852 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11d3d8-c575-4652-8b74-00ef1499c0f6 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:31.949336 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 287/568] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:31.949336 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11d3d8-c575-4652-8b74-00ef1499c0f6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:31.952390 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:31.970368 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:31.975258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea202f3-0fa5-4170-9802-8a92bba8de19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:31.976318 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea202f3-0fa5-4170-9802-8a92bba8de19 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:31.976674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea202f3-0fa5-4170-9802-8a92bba8de19 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:31.976970 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea202f3-0fa5-4170-9802-8a92bba8de19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:31.984377 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea202f3-0fa5-4170-9802-8a92bba8de19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:31.985884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea202f3-0fa5-4170-9802-8a92bba8de19 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:32.000977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-bea202f3-0fa5-4170-9802-8a92bba8de19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:32.000977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:32.000977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:32.000977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:32.000977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:32.000977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:32.000977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:32.000977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:32.000977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:32.000977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:32.000977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:32.000977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:32.000977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:15:32.000977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:15:32.000977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:15:32.001983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:15:32.001983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:32.001983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:32.001983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:32.001983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:32.001983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:32.001983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:32.001983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:32.001983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:32.001983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:32.001983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:32.001983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:32.001983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:32.001983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:32.001983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:32.001983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:32.001983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:32.002856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:15:32.002856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:15:32.002856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:15:32.002856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:15:32.002856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:15:32.002856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:15:32.002856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:32.002856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:32.002856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 282/569] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:32.003325 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea202f3-0fa5-4170-9802-8a92bba8de19 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:32.078466 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11d3d8-c575-4652-8b74-00ef1499c0f6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:32.120082 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d29373-2a71-440f-b5f5-a24d52d6b971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:32.120932 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d29373-2a71-440f-b5f5-a24d52d6b971 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:32.121160 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d29373-2a71-440f-b5f5-a24d52d6b971 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:32.121439 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d29373-2a71-440f-b5f5-a24d52d6b971 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:32.130841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea202f3-0fa5-4170-9802-8a92bba8de19 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:32.133320 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d29373-2a71-440f-b5f5-a24d52d6b971 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:32.141078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-49d29373-2a71-440f-b5f5-a24d52d6b971 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:32.141078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:32.141078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:32.141078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:32.141078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:32.141078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:32.141078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:32.141078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:32.141078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:32.141078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:32.141078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:32.141078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:32.141078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:32.141078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:32.141078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:32.141078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:32.142335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:32.142335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:32.142335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:32.142335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:32.142335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:32.142335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:32.142335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:32.142335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:32.142335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:32.142335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:32.142335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:32.142335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:32.142335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:32.142335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:32.142335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:32.142335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:32.143352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:32.143352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:32.143352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:32.143352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:32.143352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:32.143352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:32.143352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:32.143352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:32.143352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:32.143352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:32.143352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:32.143352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:32.148736 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 288/570] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 30 18:15:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:32.149538 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d29373-2a71-440f-b5f5-a24d52d6b971 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:32.170798 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89e3a8-a266-4f69-8b1a-dc3cfd7344c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:32.171467 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89e3a8-a266-4f69-8b1a-dc3cfd7344c9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:32.171750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89e3a8-a266-4f69-8b1a-dc3cfd7344c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:32.172028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89e3a8-a266-4f69-8b1a-dc3cfd7344c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:32.172758 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89e3a8-a266-4f69-8b1a-dc3cfd7344c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:32.173069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89e3a8-a266-4f69-8b1a-dc3cfd7344c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:32.174485 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89e3a8-a266-4f69-8b1a-dc3cfd7344c9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:32.186283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 283/571] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:32.187558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ce01b-5b08-4a9e-94a6-643bd92bacde None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:32.187851 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ce01b-5b08-4a9e-94a6-643bd92bacde None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:32.188134 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ce01b-5b08-4a9e-94a6-643bd92bacde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:32.205270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f46ce01b-5b08-4a9e-94a6-643bd92bacde None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:32.206521 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f46ce01b-5b08-4a9e-94a6-643bd92bacde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:32.238163 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:32.248456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:32.276691 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d29373-2a71-440f-b5f5-a24d52d6b971 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:32.288610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 284/572] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:32.290235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ce01b-5b08-4a9e-94a6-643bd92bacde None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:32.336612 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c108df5-6564-4aa6-a867-7e4fef7c45b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:32.337414 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c108df5-6564-4aa6-a867-7e4fef7c45b0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:32.337737 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c108df5-6564-4aa6-a867-7e4fef7c45b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:32.338021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c108df5-6564-4aa6-a867-7e4fef7c45b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:32.352247 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c108df5-6564-4aa6-a867-7e4fef7c45b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:32.353966 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c108df5-6564-4aa6-a867-7e4fef7c45b0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:32.372199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-2c108df5-6564-4aa6-a867-7e4fef7c45b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:32.372199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:32.372199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:32.372199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:32.372199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:32.372199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:32.372199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:32.372199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:32.372199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:32.372199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:32.372199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:32.372199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:32.372199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:32.372199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:32.372199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:32.373501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:32.373501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:32.373501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:32.373501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:32.373501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:32.373501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:32.373501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:32.373501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:32.373501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:32.373501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:32.373501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:32.373501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:32.373501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:32.373501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:32.373501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:32.373501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:32.373501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:32.374926 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:32.374926 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:32.374926 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:32.374926 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:32.374926 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 289/573] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:32.374926 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c108df5-6564-4aa6-a867-7e4fef7c45b0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:32.411622 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ce01b-5b08-4a9e-94a6-643bd92bacde None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:32.453091 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a8caf-7bd9-4f07-a287-3f6b22e061ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:32.454032 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a8caf-7bd9-4f07-a287-3f6b22e061ad None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:32.454545 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a8caf-7bd9-4f07-a287-3f6b22e061ad None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:32.455027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a8caf-7bd9-4f07-a287-3f6b22e061ad None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:32.456054 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a8caf-7bd9-4f07-a287-3f6b22e061ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:32.456373 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a8caf-7bd9-4f07-a287-3f6b22e061ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:32.457882 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a8caf-7bd9-4f07-a287-3f6b22e061ad None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:32.471518 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 285/574] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:15:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:32.472374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a8caf-7bd9-4f07-a287-3f6b22e061ad None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:32.495665 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c108df5-6564-4aa6-a867-7e4fef7c45b0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:32.501315 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:32.530288 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:32.536618 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf9b82-26b9-40fe-adf8-a3c37f549a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:32.537945 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf9b82-26b9-40fe-adf8-a3c37f549a98 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:32.538289 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf9b82-26b9-40fe-adf8-a3c37f549a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:32.538500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf9b82-26b9-40fe-adf8-a3c37f549a98 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:32.539270 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf9b82-26b9-40fe-adf8-a3c37f549a98 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:32.539589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf9b82-26b9-40fe-adf8-a3c37f549a98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:32.541008 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf9b82-26b9-40fe-adf8-a3c37f549a98 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:32.550045 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 290/575] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 30 18:15:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:32.550887 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf9b82-26b9-40fe-adf8-a3c37f549a98 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:32.596658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a8caf-7bd9-4f07-a287-3f6b22e061ad None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:32.650922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33d364-0cf6-4496-b09e-f7b0e6fa084a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:32.651758 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33d364-0cf6-4496-b09e-f7b0e6fa084a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:32.652044 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33d364-0cf6-4496-b09e-f7b0e6fa084a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:32.652326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33d364-0cf6-4496-b09e-f7b0e6fa084a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:32.660122 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff33d364-0cf6-4496-b09e-f7b0e6fa084a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:32.661798 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff33d364-0cf6-4496-b09e-f7b0e6fa084a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:32.669049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf9b82-26b9-40fe-adf8-a3c37f549a98 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:32.681063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-ff33d364-0cf6-4496-b09e-f7b0e6fa084a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:32.681063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:32.681063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:32.681063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:32.681063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:32.681063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:32.681063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:32.681063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:32.681063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:32.681063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:32.681063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:32.681063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:32.681063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:15:32.681063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:15:32.681063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:15:32.682147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:15:32.682147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:32.682147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:32.682147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:32.682147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:32.682147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:32.682147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:32.682147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:32.682147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:32.682147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:32.682147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:32.682147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:32.682147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:32.682147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:32.682147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:32.682147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:32.682147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:32.683226 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:15:32.683226 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:15:32.683226 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:15:32.683226 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:15:32.683226 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:15:32.683226 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:15:32.683226 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:32.683226 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:32.683226 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 286/576] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:32.683226 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33d364-0cf6-4496-b09e-f7b0e6fa084a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:32.713014 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b48c1-07cc-4c43-b925-eca97514db3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:32.713849 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b48c1-07cc-4c43-b925-eca97514db3f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:32.714139 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b48c1-07cc-4c43-b925-eca97514db3f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:32.714425 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b48c1-07cc-4c43-b925-eca97514db3f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:32.725705 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7b48c1-07cc-4c43-b925-eca97514db3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:32.733568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-dd7b48c1-07cc-4c43-b925-eca97514db3f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:32.733568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:32.733568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:32.733568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:32.733568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:32.733568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:32.733568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:32.733568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:32.733568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:32.733568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:32.733568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:32.733568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:32.733568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:32.733568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:32.733568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:32.733568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:32.734565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:32.734565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:32.734565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:32.734565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:32.734565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:32.734565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:32.734565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:32.734565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:32.734565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:32.734565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:32.734565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:32.734565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:32.734565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:32.734565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:32.734565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:32.734565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:32.735405 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:32.735405 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:32.735405 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:32.735405 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:32.735405 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:32.735405 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:32.735405 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:32.735405 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:32.735405 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:32.735405 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:32.735405 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:32.735405 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:32.735405 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 291/577] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 30 18:15:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:32.736141 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b48c1-07cc-4c43-b925-eca97514db3f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:32.782496 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:32.796150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33d364-0cf6-4496-b09e-f7b0e6fa084a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:32.801142 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:32.834612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360576cd-376b-4d9b-a071-187b5bd8bf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:32.835000 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360576cd-376b-4d9b-a071-187b5bd8bf6e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:32.835281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360576cd-376b-4d9b-a071-187b5bd8bf6e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:32.835559 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360576cd-376b-4d9b-a071-187b5bd8bf6e None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:32.836280 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360576cd-376b-4d9b-a071-187b5bd8bf6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:32.836615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360576cd-376b-4d9b-a071-187b5bd8bf6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:32.838045 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360576cd-376b-4d9b-a071-187b5bd8bf6e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:32.850028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 287/578] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:32.851588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360576cd-376b-4d9b-a071-187b5bd8bf6e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:32.853294 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b48c1-07cc-4c43-b925-eca97514db3f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:32.888975 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba5420-7e8f-4777-b61a-50bb0174427d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:32.889903 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba5420-7e8f-4777-b61a-50bb0174427d None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:32.890189 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba5420-7e8f-4777-b61a-50bb0174427d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:32.890473 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba5420-7e8f-4777-b61a-50bb0174427d None admin] PATH_INFO: `/v3/projects/8c7e1ee12aa14b7ab13e07822826b25b/users/b8751a4e92174d18b7b138392d2579a1/roles/1ccfbd20ff5f4d068de0fe3bf93ebe6d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:32.908841 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bba5420-7e8f-4777-b61a-50bb0174427d None admin] RBAC: Authorizing `identity:create_grant(project_id=8c7e1ee12aa14b7ab13e07822826b25b, user_id=b8751a4e92174d18b7b138392d2579a1, role_id=1ccfbd20ff5f4d068de0fe3bf93ebe6d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:32.910451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bba5420-7e8f-4777-b61a-50bb0174427d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:32.926276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 292/579] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:15:32 2022] PUT /identity/v3/projects/8c7e1ee12aa14b7ab13e07822826b25b/users/b8751a4e92174d18b7b138392d2579a1/roles/1ccfbd20ff5f4d068de0fe3bf93ebe6d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:15:32.927128 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba5420-7e8f-4777-b61a-50bb0174427d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:32.987536 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360576cd-376b-4d9b-a071-187b5bd8bf6e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:33.041204 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d69b8-6902-4e38-9bac-8464159e0d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:33.042103 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d69b8-6902-4e38-9bac-8464159e0d57 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:33.042401 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d69b8-6902-4e38-9bac-8464159e0d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:33.042687 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d69b8-6902-4e38-9bac-8464159e0d57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:33.056272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba5420-7e8f-4777-b61a-50bb0174427d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:33.057856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32d69b8-6902-4e38-9bac-8464159e0d57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:33.059448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:33.060319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32d69b8-6902-4e38-9bac-8464159e0d57 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:33.079253 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:33.091750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c32d69b8-6902-4e38-9bac-8464159e0d57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:33.091750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:33.091750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:33.091750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:33.091750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:33.091750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:33.091750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:33.091750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:33.091750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:33.091750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:33.091750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:33.091750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:33.091750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:33.091750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:33.091750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:33.093215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:33.093215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:33.093215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:33.093215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:33.093215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:33.093215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:33.093215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:33.093215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:33.093215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:33.093215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:33.093215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:33.093215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:33.093215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:33.093215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:33.093215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:33.093215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:33.093215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:33.094250 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:33.094250 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:33.094250 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:33.094250 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:33.094250 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 288/580] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:33.094250 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d69b8-6902-4e38-9bac-8464159e0d57 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:33.119073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da397d-2893-418d-beb9-16d4923a2e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:33.119790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da397d-2893-418d-beb9-16d4923a2e6a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:33.120081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da397d-2893-418d-beb9-16d4923a2e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:33.120361 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da397d-2893-418d-beb9-16d4923a2e6a None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:33.121190 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da397d-2893-418d-beb9-16d4923a2e6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:33.121503 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da397d-2893-418d-beb9-16d4923a2e6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:33.123121 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da397d-2893-418d-beb9-16d4923a2e6a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:33.142176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 293/581] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:15:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:33.142176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da397d-2893-418d-beb9-16d4923a2e6a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:33.222100 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d69b8-6902-4e38-9bac-8464159e0d57 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:33.263858 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d963f-96c3-4cc0-9641-44c377e45f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:33.264686 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d963f-96c3-4cc0-9641-44c377e45f18 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:33.264970 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d963f-96c3-4cc0-9641-44c377e45f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:33.265250 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d963f-96c3-4cc0-9641-44c377e45f18 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:33.266651 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d963f-96c3-4cc0-9641-44c377e45f18 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:33.266961 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d963f-96c3-4cc0-9641-44c377e45f18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:33.268208 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da397d-2893-418d-beb9-16d4923a2e6a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:33.268397 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d963f-96c3-4cc0-9641-44c377e45f18 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:33.279081 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 289/582] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 30 18:15:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:33.279935 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d963f-96c3-4cc0-9641-44c377e45f18 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:33.313211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d91fc9c-3bcb-4626-ab0d-b6bec269630d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:33.314048 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91fc9c-3bcb-4626-ab0d-b6bec269630d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:33.314365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91fc9c-3bcb-4626-ab0d-b6bec269630d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:33.314758 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91fc9c-3bcb-4626-ab0d-b6bec269630d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:33.323615 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d91fc9c-3bcb-4626-ab0d-b6bec269630d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:33.325111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d91fc9c-3bcb-4626-ab0d-b6bec269630d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:33.340683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-9d91fc9c-3bcb-4626-ab0d-b6bec269630d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 18:15:33.340683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:33.340683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:33.340683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:33.340683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:33.340683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:33.340683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:33.340683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:33.340683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:33.340683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:33.340683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:33.340683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:33.340683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:15:33.340683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:15:33.340683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:15:33.340683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:15:33.341729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:33.341729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:33.341729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:15:33.341729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:33.341729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:15:33.341729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:33.341729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:33.341729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:33.341729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:33.341729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:33.341729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:33.341729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:33.341729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:33.341729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:33.341729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:33.341729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:33.341729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:33.341729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:33.342666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:33.342666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:33.342666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:15:33.342666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:15:33.342666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:15:33.342666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:15:33.342666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:15:33.342666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:15:33.342666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 18:15:33.342666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:33.347013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 294/583] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:15:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:33.347653 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d91fc9c-3bcb-4626-ab0d-b6bec269630d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:33.364544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:33.388654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:33.424608 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d963f-96c3-4cc0-9641-44c377e45f18 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:33.463512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7171fc-02d8-4b13-a66e-105c93a9d0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:33.464189 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7171fc-02d8-4b13-a66e-105c93a9d0bc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:33.464479 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7171fc-02d8-4b13-a66e-105c93a9d0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:33.464762 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7171fc-02d8-4b13-a66e-105c93a9d0bc None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:33.465502 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7171fc-02d8-4b13-a66e-105c93a9d0bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:33.465915 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7171fc-02d8-4b13-a66e-105c93a9d0bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:33.467319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7171fc-02d8-4b13-a66e-105c93a9d0bc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:33.481310 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 290/584] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:33.482489 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7171fc-02d8-4b13-a66e-105c93a9d0bc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:33.491892 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d91fc9c-3bcb-4626-ab0d-b6bec269630d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:33.533324 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096cfbf2-ac1b-45b1-9e08-5428e0fa55a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:33.534027 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cfbf2-ac1b-45b1-9e08-5428e0fa55a9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:33.534325 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cfbf2-ac1b-45b1-9e08-5428e0fa55a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:33.534612 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cfbf2-ac1b-45b1-9e08-5428e0fa55a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:33.535464 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096cfbf2-ac1b-45b1-9e08-5428e0fa55a9 None admin] RBAC: Adding query filter params (role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d, scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b, user.id=4bd521b9e93f4b3b8af64d46f36c019e) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:33.535781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096cfbf2-ac1b-45b1-9e08-5428e0fa55a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:33.537518 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096cfbf2-ac1b-45b1-9e08-5428e0fa55a9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:33.553216 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 295/585] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:15:33 2022] GET /identity/v3/role_assignments?user.id=4bd521b9e93f4b3b8af64d46f36c019e&scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b&role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:33.553216 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096cfbf2-ac1b-45b1-9e08-5428e0fa55a9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:33.624655 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7171fc-02d8-4b13-a66e-105c93a9d0bc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:33.661427 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:33.684876 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e671cf-9591-4640-a7f6-935575573444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:33.685942 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e671cf-9591-4640-a7f6-935575573444 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:33.686373 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e671cf-9591-4640-a7f6-935575573444 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:33.686739 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e671cf-9591-4640-a7f6-935575573444 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:33.702640 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e671cf-9591-4640-a7f6-935575573444 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:33.704544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e671cf-9591-4640-a7f6-935575573444 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:33.705250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:33.723772 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096cfbf2-ac1b-45b1-9e08-5428e0fa55a9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:33.731740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-26e671cf-9591-4640-a7f6-935575573444 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:33.731740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:33.731740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:33.731740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:33.731740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:33.731740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:33.731740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:33.731740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:33.731740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:33.731740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:33.731740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:33.731740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:33.731740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:33.731740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:33.731740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:33.732835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:33.732835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:33.732835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:33.732835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:33.732835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:33.732835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:33.732835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:33.732835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:33.732835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:33.732835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:33.732835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:33.732835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:33.732835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:33.732835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:33.732835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:33.732835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:33.732835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:33.733784 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:33.733784 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:33.733784 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:33.733784 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:33.733784 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 291/586] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:33.734084 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e671cf-9591-4640-a7f6-935575573444 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:33.781550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae00e3e-e8ab-4ec4-a94f-531d4219d4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:33.782502 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae00e3e-e8ab-4ec4-a94f-531d4219d4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:33.782959 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae00e3e-e8ab-4ec4-a94f-531d4219d4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:33.783400 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae00e3e-e8ab-4ec4-a94f-531d4219d4d8 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:33.784419 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae00e3e-e8ab-4ec4-a94f-531d4219d4d8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:33.784879 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae00e3e-e8ab-4ec4-a94f-531d4219d4d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:33.787709 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae00e3e-e8ab-4ec4-a94f-531d4219d4d8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:33.814398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 296/587] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Jun 30 18:15:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:33.815367 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae00e3e-e8ab-4ec4-a94f-531d4219d4d8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:33.863886 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e671cf-9591-4640-a7f6-935575573444 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:33.904065 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f6864-55b3-4c0f-8122-868d22e445d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:33.905223 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f6864-55b3-4c0f-8122-868d22e445d8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:33.905742 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f6864-55b3-4c0f-8122-868d22e445d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:33.906249 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f6864-55b3-4c0f-8122-868d22e445d8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:33.915977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084f6864-55b3-4c0f-8122-868d22e445d8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:33.917996 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084f6864-55b3-4c0f-8122-868d22e445d8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:33.929974 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae00e3e-e8ab-4ec4-a94f-531d4219d4d8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:33.935567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-084f6864-55b3-4c0f-8122-868d22e445d8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 18:15:33.935567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:33.935567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:33.935567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:33.935567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:33.935567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:33.935567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:33.935567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:33.935567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:33.935567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:33.935567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:33.935567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:33.935567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:15:33.935567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:15:33.935567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:15:33.935567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:15:33.936540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:33.936540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:33.936540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:15:33.936540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:33.936540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:15:33.936540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:33.936540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:33.936540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:33.936540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:33.936540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:33.936540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:33.936540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:33.936540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:33.936540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:33.936540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:33.936540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:33.936540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:33.936540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:33.937470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:33.937470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:33.937470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:15:33.937470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:15:33.937470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:15:33.937470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:15:33.937470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:15:33.937470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:15:33.937470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 18:15:33.937470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:33.940260 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 292/588] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:15:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:33.941332 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f6864-55b3-4c0f-8122-868d22e445d8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:33.970270 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b8484-e07d-488b-a471-ced440bb7125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:33.970953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b8484-e07d-488b-a471-ced440bb7125 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:33.971240 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b8484-e07d-488b-a471-ced440bb7125 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:33.971631 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b8484-e07d-488b-a471-ced440bb7125 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:33.972313 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b8484-e07d-488b-a471-ced440bb7125 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:33.972672 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b8484-e07d-488b-a471-ced440bb7125 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:33.974239 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b8484-e07d-488b-a471-ced440bb7125 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:33.985981 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:33.989784 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 297/589] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:33.991512 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b8484-e07d-488b-a471-ced440bb7125 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:34.001495 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:34.067974 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f6864-55b3-4c0f-8122-868d22e445d8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:34.109517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7950f-d7ab-4d36-98ad-b185195ecbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:34.110814 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7950f-d7ab-4d36-98ad-b185195ecbd6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:34.111096 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b8484-e07d-488b-a471-ced440bb7125 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:34.111531 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7950f-d7ab-4d36-98ad-b185195ecbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:34.112006 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7950f-d7ab-4d36-98ad-b185195ecbd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:34.125194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7950f-d7ab-4d36-98ad-b185195ecbd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:34.127075 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7950f-d7ab-4d36-98ad-b185195ecbd6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:34.149307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-71e7950f-d7ab-4d36-98ad-b185195ecbd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:34.149307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:34.149307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:34.149307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:34.149307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:34.149307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:34.149307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:34.149307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:34.149307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:34.149307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:34.149307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:34.149307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:34.149307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:34.149307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:34.149307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:34.150443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:34.150443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:34.150443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:34.150443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:34.150443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:34.150443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:34.150443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:34.150443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:34.150443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:34.150443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:34.150443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:34.150443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:34.150443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:34.150443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:34.150443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:34.150443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:34.150443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:34.151534 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:34.151534 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:34.151534 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:34.151534 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:34.152751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df1aa3-daf6-4c3e-9fa0-b832d23da41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:34.153722 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df1aa3-daf6-4c3e-9fa0-b832d23da41b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:34.155453 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df1aa3-daf6-4c3e-9fa0-b832d23da41b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:34.155453 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df1aa3-daf6-4c3e-9fa0-b832d23da41b None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:34.155453 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df1aa3-daf6-4c3e-9fa0-b832d23da41b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:34.155453 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df1aa3-daf6-4c3e-9fa0-b832d23da41b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:34.156168 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 293/590] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:34.157266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7950f-d7ab-4d36-98ad-b185195ecbd6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:34.157478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df1aa3-daf6-4c3e-9fa0-b832d23da41b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:34.180151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 298/591] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Jun 30 18:15:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:34.181226 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df1aa3-daf6-4c3e-9fa0-b832d23da41b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:34.251478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:34.259064 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:34.277764 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7950f-d7ab-4d36-98ad-b185195ecbd6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:34.307199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df1aa3-daf6-4c3e-9fa0-b832d23da41b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:34.317077 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37eb3a-ce93-4403-b155-b578e02fdcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:34.317915 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37eb3a-ce93-4403-b155-b578e02fdcb0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:34.318271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37eb3a-ce93-4403-b155-b578e02fdcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:34.318791 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37eb3a-ce93-4403-b155-b578e02fdcb0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:34.327710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37eb3a-ce93-4403-b155-b578e02fdcb0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:34.329190 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37eb3a-ce93-4403-b155-b578e02fdcb0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:34.346841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-df37eb3a-ce93-4403-b155-b578e02fdcb0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 18:15:34.346841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:34.346841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:34.346841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:34.346841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:34.346841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:34.346841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:34.346841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:34.346841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:34.346841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:34.346841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:34.346841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:34.346841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:15:34.346841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:15:34.346841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:15:34.346841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:15:34.347842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:34.347842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:34.347842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:15:34.347842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:34.347842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:15:34.347842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:34.347842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:34.347842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:34.347842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:34.347842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:34.347842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:34.347842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:34.347842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:34.347842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:34.347842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:34.347842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:34.347842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:34.347842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:34.348787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:34.348787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:34.348787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:15:34.348787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:15:34.348787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:15:34.348787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:15:34.348787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:15:34.348787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:15:34.348787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 18:15:34.348787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:34.348787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 294/592] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 30 18:15:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:34.349396 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37eb3a-ce93-4403-b155-b578e02fdcb0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:34.352079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5741124-dfe3-46b5-ae6d-1f215d14504b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:34.353039 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5741124-dfe3-46b5-ae6d-1f215d14504b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:34.353531 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5741124-dfe3-46b5-ae6d-1f215d14504b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:34.354064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5741124-dfe3-46b5-ae6d-1f215d14504b None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:34.355000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5741124-dfe3-46b5-ae6d-1f215d14504b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:34.355523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5741124-dfe3-46b5-ae6d-1f215d14504b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:34.357143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5741124-dfe3-46b5-ae6d-1f215d14504b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:34.370666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 299/593] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:34.371871 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5741124-dfe3-46b5-ae6d-1f215d14504b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:34.466903 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37eb3a-ce93-4403-b155-b578e02fdcb0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:34.486223 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5741124-dfe3-46b5-ae6d-1f215d14504b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:34.509376 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6515b56-d9af-4f4f-b78e-bde79fc3141b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:34.510193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6515b56-d9af-4f4f-b78e-bde79fc3141b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:34.510482 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6515b56-d9af-4f4f-b78e-bde79fc3141b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:34.510761 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6515b56-d9af-4f4f-b78e-bde79fc3141b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:34.518928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:34.523304 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6515b56-d9af-4f4f-b78e-bde79fc3141b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:34.524925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6515b56-d9af-4f4f-b78e-bde79fc3141b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:34.533068 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:34.539856 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623670d0-8c1c-45d2-82f2-2227dc5ca8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:34.540834 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623670d0-8c1c-45d2-82f2-2227dc5ca8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:34.541271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623670d0-8c1c-45d2-82f2-2227dc5ca8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:34.541757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623670d0-8c1c-45d2-82f2-2227dc5ca8d9 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:34.542951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623670d0-8c1c-45d2-82f2-2227dc5ca8d9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:34.543412 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623670d0-8c1c-45d2-82f2-2227dc5ca8d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:34.545246 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623670d0-8c1c-45d2-82f2-2227dc5ca8d9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:34.549135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f6515b56-d9af-4f4f-b78e-bde79fc3141b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:34.549135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:34.549135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:34.549135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:34.549135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:34.549135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:34.549135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:34.549135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:34.549135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:34.549135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:34.549135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:34.549135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:34.549135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:34.549135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:34.549135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:34.550239 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:34.550239 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:34.550239 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:34.550239 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:34.550239 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:34.550239 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:34.550239 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:34.550239 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:34.550239 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:34.550239 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:34.550239 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:34.550239 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:34.550239 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:34.550239 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:34.550239 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:34.550239 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:34.550239 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:34.551254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:34.551254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:34.551254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:34.551254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:34.551254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 295/594] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:34.552074 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6515b56-d9af-4f4f-b78e-bde79fc3141b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:34.567884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 300/595] 10.222.0.33 () {56 vars in 1139 bytes} [Thu Jun 30 18:15:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:34.569943 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623670d0-8c1c-45d2-82f2-2227dc5ca8d9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:34.661475 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6515b56-d9af-4f4f-b78e-bde79fc3141b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:34.680580 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623670d0-8c1c-45d2-82f2-2227dc5ca8d9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:34.700323 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61cef2-9a4b-468a-95a4-f7209b831ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:34.701215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61cef2-9a4b-468a-95a4-f7209b831ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:34.701403 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61cef2-9a4b-468a-95a4-f7209b831ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:34.701718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61cef2-9a4b-468a-95a4-f7209b831ea1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:34.709786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61cef2-9a4b-468a-95a4-f7209b831ea1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:34.711560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61cef2-9a4b-468a-95a4-f7209b831ea1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:34.720593 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39a108-9277-4d19-b1d4-ad026b27f5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:34.721914 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39a108-9277-4d19-b1d4-ad026b27f5ce None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:34.722381 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39a108-9277-4d19-b1d4-ad026b27f5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:34.722837 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39a108-9277-4d19-b1d4-ad026b27f5ce None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:34.723736 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b39a108-9277-4d19-b1d4-ad026b27f5ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:34.724224 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b39a108-9277-4d19-b1d4-ad026b27f5ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:34.726513 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b39a108-9277-4d19-b1d4-ad026b27f5ce None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:34.727544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-1c61cef2-9a4b-468a-95a4-f7209b831ea1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:34.727544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:34.727544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:34.727544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:34.727544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:34.727544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:34.727544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:34.727544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:34.727544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:34.727544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:34.727544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:34.727544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:34.727544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:15:34.727544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:15:34.727544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:15:34.728400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:15:34.728400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:34.728400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:34.728400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:34.728400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:34.728400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:34.728400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:34.728400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:34.728400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:34.728400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:34.728400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:34.728400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:34.728400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:34.728400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:34.728400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:34.728400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:34.728400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:34.729295 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:15:34.729295 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:15:34.729295 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:15:34.729295 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:15:34.729295 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:15:34.729295 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:15:34.729295 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:34.729295 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:34.729295 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 296/596] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:34.729966 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61cef2-9a4b-468a-95a4-f7209b831ea1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:34.743514 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 301/597] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:34.744716 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39a108-9277-4d19-b1d4-ad026b27f5ce None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:34.793334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:34.795251 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:34.840857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61cef2-9a4b-468a-95a4-f7209b831ea1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:34.861294 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39a108-9277-4d19-b1d4-ad026b27f5ce None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:34.878921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cddc5d-db21-4e9e-9a30-36a782ac726b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:34.879945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cddc5d-db21-4e9e-9a30-36a782ac726b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:34.880176 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cddc5d-db21-4e9e-9a30-36a782ac726b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:34.880350 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cddc5d-db21-4e9e-9a30-36a782ac726b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:34.892576 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cddc5d-db21-4e9e-9a30-36a782ac726b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:34.894150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cddc5d-db21-4e9e-9a30-36a782ac726b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:34.908607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94406c-a842-4f4d-9098-db10ced6ad27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:34.909658 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94406c-a842-4f4d-9098-db10ced6ad27 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:34.910183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94406c-a842-4f4d-9098-db10ced6ad27 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:34.910669 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94406c-a842-4f4d-9098-db10ced6ad27 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:34.911910 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e94406c-a842-4f4d-9098-db10ced6ad27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:34.912960 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e94406c-a842-4f4d-9098-db10ced6ad27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:34.914697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e94406c-a842-4f4d-9098-db10ced6ad27 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:34.916884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b6cddc5d-db21-4e9e-9a30-36a782ac726b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:34.916884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:34.916884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:34.916884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:34.916884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:34.916884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:34.916884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:34.916884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:34.916884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:34.916884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:34.916884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:34.916884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:34.916884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:34.916884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:34.916884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:34.917810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:34.917810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:34.917810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:34.917810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:34.917810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:34.917810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:34.917810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:34.917810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:34.917810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:34.917810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:34.917810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:34.917810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:34.917810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:34.917810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:34.917810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:34.917810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:34.917810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:34.918727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:34.918727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:34.918727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:34.918727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:34.918727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 297/598] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:34.919705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cddc5d-db21-4e9e-9a30-36a782ac726b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:34.932221 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 302/599] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:15:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:34.933532 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94406c-a842-4f4d-9098-db10ced6ad27 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:35.040701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cddc5d-db21-4e9e-9a30-36a782ac726b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:35.051512 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94406c-a842-4f4d-9098-db10ced6ad27 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:35.052100 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:35.056043 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:35.079910 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23164965-2e27-434c-86b6-f3ec15b36709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:35.081101 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23164965-2e27-434c-86b6-f3ec15b36709 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:35.081526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23164965-2e27-434c-86b6-f3ec15b36709 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:35.082018 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23164965-2e27-434c-86b6-f3ec15b36709 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:35.091233 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23164965-2e27-434c-86b6-f3ec15b36709 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:35.092725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23164965-2e27-434c-86b6-f3ec15b36709 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:35.103975 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384ab66-adb1-4c43-8c95-6608c1a297eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:35.105473 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384ab66-adb1-4c43-8c95-6608c1a297eb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:35.106063 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384ab66-adb1-4c43-8c95-6608c1a297eb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:35.106549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384ab66-adb1-4c43-8c95-6608c1a297eb None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:35.107489 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384ab66-adb1-4c43-8c95-6608c1a297eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:35.108158 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384ab66-adb1-4c43-8c95-6608c1a297eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:35.109472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-23164965-2e27-434c-86b6-f3ec15b36709 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:35.109472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:35.109472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:35.109472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:35.109472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:35.109472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:35.109472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:35.109472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:35.109472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:35.109472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:35.109472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:35.109472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:35.109472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:15:35.109472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:15:35.109472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:15:35.110379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:15:35.110379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:35.110379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:35.110379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:35.110379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:35.110379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:35.110379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:35.110379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:35.110379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:35.110379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:35.110379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:35.110379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:35.110379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:35.110379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:35.110379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:35.110379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:35.110379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:35.111264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:15:35.111264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:15:35.111264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:15:35.111264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:15:35.111264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:15:35.111264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:15:35.111264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:35.111264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:35.111264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 298/600] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:35.112183 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23164965-2e27-434c-86b6-f3ec15b36709 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:35.112765 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384ab66-adb1-4c43-8c95-6608c1a297eb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:35.127402 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 303/601] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:35.128917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384ab66-adb1-4c43-8c95-6608c1a297eb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:35.231701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23164965-2e27-434c-86b6-f3ec15b36709 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:35.277094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384ab66-adb1-4c43-8c95-6608c1a297eb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:35.278157 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45786b2c-b042-46d5-a76e-39d1a5daf9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:35.279057 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45786b2c-b042-46d5-a76e-39d1a5daf9d1 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:35.279341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45786b2c-b042-46d5-a76e-39d1a5daf9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:35.279616 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45786b2c-b042-46d5-a76e-39d1a5daf9d1 None admin] PATH_INFO: `/v3/projects/8c7e1ee12aa14b7ab13e07822826b25b/users/37a9fea702cd405ea9fc14ec2c689648/roles/1ccfbd20ff5f4d068de0fe3bf93ebe6d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:35.299018 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45786b2c-b042-46d5-a76e-39d1a5daf9d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c7e1ee12aa14b7ab13e07822826b25b, user_id=37a9fea702cd405ea9fc14ec2c689648, role_id=1ccfbd20ff5f4d068de0fe3bf93ebe6d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:35.300495 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45786b2c-b042-46d5-a76e-39d1a5daf9d1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:35.315251 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 299/602] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:15:35 2022] PUT /identity/v3/projects/8c7e1ee12aa14b7ab13e07822826b25b/users/37a9fea702cd405ea9fc14ec2c689648/roles/1ccfbd20ff5f4d068de0fe3bf93ebe6d => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:15:35.315994 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45786b2c-b042-46d5-a76e-39d1a5daf9d1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:35.330875 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592cb20-b37e-4fe5-ba78-1ec79c88c5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:35.332214 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592cb20-b37e-4fe5-ba78-1ec79c88c5cf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:35.332694 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592cb20-b37e-4fe5-ba78-1ec79c88c5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:35.333434 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592cb20-b37e-4fe5-ba78-1ec79c88c5cf None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:35.335381 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592cb20-b37e-4fe5-ba78-1ec79c88c5cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:35.336117 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592cb20-b37e-4fe5-ba78-1ec79c88c5cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:35.338490 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592cb20-b37e-4fe5-ba78-1ec79c88c5cf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:35.343196 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:35.356625 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 304/603] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:15:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:35.360006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c7cf0-7d62-4dda-9bcb-94c35bbbdfd2 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:35.360694 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c7cf0-7d62-4dda-9bcb-94c35bbbdfd2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:35.361301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c7cf0-7d62-4dda-9bcb-94c35bbbdfd2 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:35.369399 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 305/604] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:35 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:35.372606 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c7cf0-7d62-4dda-9bcb-94c35bbbdfd2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:35.410161 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:35.462838 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45786b2c-b042-46d5-a76e-39d1a5daf9d1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:35.505899 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d570e4-b2ed-4c3d-8693-f74c82c0a4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:35.506872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d570e4-b2ed-4c3d-8693-f74c82c0a4fb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:35.507259 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d570e4-b2ed-4c3d-8693-f74c82c0a4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:35.508210 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d570e4-b2ed-4c3d-8693-f74c82c0a4fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:35.515894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d570e4-b2ed-4c3d-8693-f74c82c0a4fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:35.517487 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d570e4-b2ed-4c3d-8693-f74c82c0a4fb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:35.522819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c7cf0-7d62-4dda-9bcb-94c35bbbdfd2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:35.533516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e4d570e4-b2ed-4c3d-8693-f74c82c0a4fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:35.533516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:35.533516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:35.533516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:35.533516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:35.533516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:35.533516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:35.533516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:35.533516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:35.533516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:35.533516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:35.533516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:35.533516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:15:35.533516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:15:35.533516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:15:35.534725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:15:35.534725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:35.534725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:35.534725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:35.534725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:35.534725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:35.534725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:35.534725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:35.534725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:35.534725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:35.534725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:35.534725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:35.534725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:35.534725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:35.534725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:35.534725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:35.534725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:35.535848 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:15:35.535848 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:15:35.535848 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:15:35.535848 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:15:35.535848 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:15:35.535848 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:15:35.535848 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:35.535848 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:35.538267 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 300/605] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:35.539827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a92630-0888-4579-b7e2-1559c9cb095e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:35.540231 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a92630-0888-4579-b7e2-1559c9cb095e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:35.540604 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a92630-0888-4579-b7e2-1559c9cb095e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:35.559214 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-49a92630-0888-4579-b7e2-1559c9cb095e None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:35.560502 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-49a92630-0888-4579-b7e2-1559c9cb095e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:35.562445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3046dd68-d5ae-4b03-8849-6ed60c862707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:35.563343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046dd68-d5ae-4b03-8849-6ed60c862707 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:35.563629 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046dd68-d5ae-4b03-8849-6ed60c862707 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:35.563907 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046dd68-d5ae-4b03-8849-6ed60c862707 None admin] PATH_INFO: `/v3/projects/8c7e1ee12aa14b7ab13e07822826b25b/users/92c70ce830ea4dd981716d622b16d595/roles/1ccfbd20ff5f4d068de0fe3bf93ebe6d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:35.579665 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3046dd68-d5ae-4b03-8849-6ed60c862707 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c7e1ee12aa14b7ab13e07822826b25b, user_id=92c70ce830ea4dd981716d622b16d595, role_id=1ccfbd20ff5f4d068de0fe3bf93ebe6d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:35.581047 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3046dd68-d5ae-4b03-8849-6ed60c862707 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:35.595609 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 306/606] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:15:35 2022] PUT /identity/v3/projects/8c7e1ee12aa14b7ab13e07822826b25b/users/92c70ce830ea4dd981716d622b16d595/roles/1ccfbd20ff5f4d068de0fe3bf93ebe6d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:15:35.596525 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3046dd68-d5ae-4b03-8849-6ed60c862707 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:35.641730 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 301/607] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:35.661800 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a92630-0888-4579-b7e2-1559c9cb095e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:35.674574 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:35.702778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:35.713444 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3046dd68-d5ae-4b03-8849-6ed60c862707 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:35.769794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e71b43-27ac-478d-a899-95b5d147226b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:35.769794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e71b43-27ac-478d-a899-95b5d147226b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:35.769794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e71b43-27ac-478d-a899-95b5d147226b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:35.769794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e71b43-27ac-478d-a899-95b5d147226b None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:35.770632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e71b43-27ac-478d-a899-95b5d147226b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:35.770632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e71b43-27ac-478d-a899-95b5d147226b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:35.770632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e71b43-27ac-478d-a899-95b5d147226b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:35.785127 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 307/608] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:15:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:35.792182 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e71b43-27ac-478d-a899-95b5d147226b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:35.804778 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a92630-0888-4579-b7e2-1559c9cb095e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:35.845292 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80aa7eb-0c2d-4f31-b425-864a8f1de815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:35.846561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80aa7eb-0c2d-4f31-b425-864a8f1de815 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:35.846852 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80aa7eb-0c2d-4f31-b425-864a8f1de815 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:35.847131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80aa7eb-0c2d-4f31-b425-864a8f1de815 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:35.862447 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80aa7eb-0c2d-4f31-b425-864a8f1de815 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:35.872493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c80aa7eb-0c2d-4f31-b425-864a8f1de815 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:35.872493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:35.872493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:35.872493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:35.872493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:35.872493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:35.872493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:35.872493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:35.872493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:35.872493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:35.872493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:35.872493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:35.872493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:35.872493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:35.872493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:35.872493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:35.873458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:35.873458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:35.873458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:35.873458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:35.873458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:35.873458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:35.873458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:35.873458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:35.873458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:35.873458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:35.873458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:35.873458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:35.873458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:35.873458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:35.873458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:35.873458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:35.874656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:35.874656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:35.874656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:35.874656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:35.874656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:35.874656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:35.874656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:35.874656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:35.874656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:35.874656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:35.874656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:35.874656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:35.874656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 302/609] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 30 18:15:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:15:35.880027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80aa7eb-0c2d-4f31-b425-864a8f1de815 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:35.937254 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e71b43-27ac-478d-a899-95b5d147226b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:35.963293 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:35.967151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:35.972586 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34da8a-7d51-4c8b-88cc-c978b9149b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:35.973886 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34da8a-7d51-4c8b-88cc-c978b9149b35 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:35.974997 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34da8a-7d51-4c8b-88cc-c978b9149b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:35.975327 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34da8a-7d51-4c8b-88cc-c978b9149b35 None admin] PATH_INFO: `/v3/projects/8c7e1ee12aa14b7ab13e07822826b25b/users/1d8c54102fb340859ed0acc3189fe376/roles/1ccfbd20ff5f4d068de0fe3bf93ebe6d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:35.991610 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34da8a-7d51-4c8b-88cc-c978b9149b35 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c7e1ee12aa14b7ab13e07822826b25b, user_id=1d8c54102fb340859ed0acc3189fe376, role_id=1ccfbd20ff5f4d068de0fe3bf93ebe6d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:35.993162 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34da8a-7d51-4c8b-88cc-c978b9149b35 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:36.003656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80aa7eb-0c2d-4f31-b425-864a8f1de815 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:36.012360 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 308/610] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:15:35 2022] PUT /identity/v3/projects/8c7e1ee12aa14b7ab13e07822826b25b/users/1d8c54102fb340859ed0acc3189fe376/roles/1ccfbd20ff5f4d068de0fe3bf93ebe6d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:15:36.014238 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfcdfc-2fb0-4918-b9a4-ee885b95b667 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:36.014772 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfcdfc-2fb0-4918-b9a4-ee885b95b667 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:36.015251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfcdfc-2fb0-4918-b9a4-ee885b95b667 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:36.017340 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 309/611] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:36.023163 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99739bd-7d0f-44d7-9a9d-a0339d83f569 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:36.023724 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99739bd-7d0f-44d7-9a9d-a0339d83f569 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:36.024240 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99739bd-7d0f-44d7-9a9d-a0339d83f569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:36.048210 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d99739bd-7d0f-44d7-9a9d-a0339d83f569 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:36.050114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d99739bd-7d0f-44d7-9a9d-a0339d83f569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:36.071858 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec092c-1c52-470d-b8be-93834098e284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:36.072618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec092c-1c52-470d-b8be-93834098e284 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:36.072978 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec092c-1c52-470d-b8be-93834098e284 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:36.073392 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec092c-1c52-470d-b8be-93834098e284 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:36.074301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec092c-1c52-470d-b8be-93834098e284 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:36.074777 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec092c-1c52-470d-b8be-93834098e284 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:36.076407 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec092c-1c52-470d-b8be-93834098e284 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:36.102336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 303/612] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 30 18:15:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:15:36.105484 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec092c-1c52-470d-b8be-93834098e284 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:36.152311 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 310/613] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:36.187161 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99739bd-7d0f-44d7-9a9d-a0339d83f569 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:36.250997 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec092c-1c52-470d-b8be-93834098e284 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:36.302795 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:36.321298 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca14f0e-74a4-4c4e-a24e-91603a890b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:36.323220 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca14f0e-74a4-4c4e-a24e-91603a890b7a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:36.324100 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca14f0e-74a4-4c4e-a24e-91603a890b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:36.324993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca14f0e-74a4-4c4e-a24e-91603a890b7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:36.349415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca14f0e-74a4-4c4e-a24e-91603a890b7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:36.352822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca14f0e-74a4-4c4e-a24e-91603a890b7a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:36.379564 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99739bd-7d0f-44d7-9a9d-a0339d83f569 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:36.380558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-dca14f0e-74a4-4c4e-a24e-91603a890b7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:36.380558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:36.380558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:36.380558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:36.380558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:36.380558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:36.380558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:36.380558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:36.380558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:36.380558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:36.380558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:36.380558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:36.380558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:36.380558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:36.380558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:36.382867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:36.382867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:36.382867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:36.382867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:36.382867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:36.382867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:36.382867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:36.382867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:36.382867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:36.382867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:36.382867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:36.382867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:36.382867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:36.382867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:36.382867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:36.382867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:36.382867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:36.385471 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:36.385471 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:36.385471 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:36.385471 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:36.385471 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 304/614] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:15:36.391690 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca14f0e-74a4-4c4e-a24e-91603a890b7a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:36.395864 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:36.421679 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1595019-c052-4bc8-b577-e9f2e848884a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:36.423683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1595019-c052-4bc8-b577-e9f2e848884a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:36.423683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1595019-c052-4bc8-b577-e9f2e848884a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:36.423683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1595019-c052-4bc8-b577-e9f2e848884a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:36.437066 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1595019-c052-4bc8-b577-e9f2e848884a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:36.444519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-c1595019-c052-4bc8-b577-e9f2e848884a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:36.444519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:36.444519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:36.444519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:36.444519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:36.444519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:36.444519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:36.444519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:36.444519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:36.444519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:36.444519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:36.444519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:36.444519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:36.444519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:36.444519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:36.444519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:36.446630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:36.446630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:36.446630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:36.446630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:36.446630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:36.446630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:36.446630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:36.446630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:36.446630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:36.446630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:36.446630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:36.446630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:36.446630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:36.446630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:36.446630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:36.446630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:36.448385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:36.448385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:36.448385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:36.448385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:36.448385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:36.448385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:36.448385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:36.448385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:36.448385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:36.448385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:36.448385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:36.448385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:36.448385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 311/615] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 30 18:15:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:15:36.459892 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1595019-c052-4bc8-b577-e9f2e848884a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:36.628721 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca14f0e-74a4-4c4e-a24e-91603a890b7a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:36.659924 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1595019-c052-4bc8-b577-e9f2e848884a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:36.700403 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c0bb8-66b4-4554-89e5-d852e5168256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:36.701094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c0bb8-66b4-4554-89e5-d852e5168256 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:36.701381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c0bb8-66b4-4554-89e5-d852e5168256 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:36.701685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c0bb8-66b4-4554-89e5-d852e5168256 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:36.702432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c0bb8-66b4-4554-89e5-d852e5168256 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:36.702739 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c0bb8-66b4-4554-89e5-d852e5168256 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:36.704151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c0bb8-66b4-4554-89e5-d852e5168256 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:36.736560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 305/616] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:36.737808 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c27d70-a592-42a7-b5cb-4c6c989acee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:36.738615 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c27d70-a592-42a7-b5cb-4c6c989acee7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:36.738914 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c27d70-a592-42a7-b5cb-4c6c989acee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:36.739228 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c27d70-a592-42a7-b5cb-4c6c989acee7 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:36.740700 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c27d70-a592-42a7-b5cb-4c6c989acee7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:36.741033 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c27d70-a592-42a7-b5cb-4c6c989acee7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:36.742631 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c27d70-a592-42a7-b5cb-4c6c989acee7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:36.742755 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c0bb8-66b4-4554-89e5-d852e5168256 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:36.752539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 312/617] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 30 18:15:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:15:36.758596 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c27d70-a592-42a7-b5cb-4c6c989acee7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:36.821966 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:36.842310 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:36.900056 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c27d70-a592-42a7-b5cb-4c6c989acee7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:36.944884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c0bb8-66b4-4554-89e5-d852e5168256 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:36.954008 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9cb49-12cb-4f20-97af-380cceaaf881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:36.954891 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9cb49-12cb-4f20-97af-380cceaaf881 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:36.955236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9cb49-12cb-4f20-97af-380cceaaf881 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:36.955526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9cb49-12cb-4f20-97af-380cceaaf881 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:36.983948 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9cb49-12cb-4f20-97af-380cceaaf881 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:36.985486 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9cb49-12cb-4f20-97af-380cceaaf881 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:36.995709 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c93ef0-6883-4fb7-9ef6-9e4d2bb280ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:36.996550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c93ef0-6883-4fb7-9ef6-9e4d2bb280ca None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:36.996944 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c93ef0-6883-4fb7-9ef6-9e4d2bb280ca None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:36.997349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c93ef0-6883-4fb7-9ef6-9e4d2bb280ca None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:37.036768 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c93ef0-6883-4fb7-9ef6-9e4d2bb280ca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:37.040276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c93ef0-6883-4fb7-9ef6-9e4d2bb280ca None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:37.060998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f5b9cb49-12cb-4f20-97af-380cceaaf881 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:37.060998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:37.060998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:37.060998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:37.060998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:37.060998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:37.060998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:37.060998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:37.060998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:37.060998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:37.060998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:37.060998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:37.060998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:37.060998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:37.060998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:37.062778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:37.062778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:37.062778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:37.062778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:37.062778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:37.062778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:37.062778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:37.062778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:37.062778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:37.062778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:37.062778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:37.062778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:37.062778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:37.062778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:37.062778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:37.062778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:37.062778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:37.064549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:37.064549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:37.064549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:37.064549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:37.069621 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 313/618] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:37.072453 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9cb49-12cb-4f20-97af-380cceaaf881 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:37.096737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b4c93ef0-6883-4fb7-9ef6-9e4d2bb280ca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 18:15:37.096737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:37.096737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:37.096737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:37.096737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:37.096737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:37.096737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:37.096737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:37.096737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:37.096737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:37.096737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:37.096737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:37.096737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:15:37.096737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:15:37.096737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:15:37.096737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:15:37.099230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:37.099230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:37.099230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:15:37.099230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:37.099230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:15:37.099230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:37.099230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:37.099230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:37.099230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:37.099230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:37.099230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:37.099230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:37.099230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:37.099230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:37.099230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:37.099230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:37.099230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:37.099230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:37.101633 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:37.101633 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:37.101633 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:15:37.101633 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:15:37.101633 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:15:37.101633 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:15:37.101633 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:15:37.101633 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:15:37.101633 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 18:15:37.101633 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:37.108375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 306/619] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 30 18:15:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:37.110055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c93ef0-6883-4fb7-9ef6-9e4d2bb280ca None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:37.324546 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:37.355923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c93ef0-6883-4fb7-9ef6-9e4d2bb280ca None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:37.362852 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:37.372462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9cb49-12cb-4f20-97af-380cceaaf881 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:37.429211 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24791da4-3404-4df4-9e49-41cbc3e4bf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:37.429502 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24791da4-3404-4df4-9e49-41cbc3e4bf14 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:37.429881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24791da4-3404-4df4-9e49-41cbc3e4bf14 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:37.430065 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24791da4-3404-4df4-9e49-41cbc3e4bf14 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:37.430907 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24791da4-3404-4df4-9e49-41cbc3e4bf14 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:37.431216 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24791da4-3404-4df4-9e49-41cbc3e4bf14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:37.432701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24791da4-3404-4df4-9e49-41cbc3e4bf14 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:37.459421 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 307/620] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 30 18:15:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:37.463977 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e046d-b75d-46d2-a6b4-753f8e12f672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:37.465422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24791da4-3404-4df4-9e49-41cbc3e4bf14 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:37.466744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e046d-b75d-46d2-a6b4-753f8e12f672 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:37.467035 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e046d-b75d-46d2-a6b4-753f8e12f672 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:37.467397 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e046d-b75d-46d2-a6b4-753f8e12f672 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:37.468161 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e046d-b75d-46d2-a6b4-753f8e12f672 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:37.468474 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e046d-b75d-46d2-a6b4-753f8e12f672 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:37.469938 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e046d-b75d-46d2-a6b4-753f8e12f672 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:37.511137 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 314/621] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:15:37.517334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e046d-b75d-46d2-a6b4-753f8e12f672 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:37.635930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24791da4-3404-4df4-9e49-41cbc3e4bf14 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:37.675303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacfcd8-8598-42d0-ab7d-97a33b107342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:37.675964 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacfcd8-8598-42d0-ab7d-97a33b107342 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:37.676249 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacfcd8-8598-42d0-ab7d-97a33b107342 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:37.676531 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacfcd8-8598-42d0-ab7d-97a33b107342 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:37.710673 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:37.754324 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcacfcd8-8598-42d0-ab7d-97a33b107342 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:37.756544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcacfcd8-8598-42d0-ab7d-97a33b107342 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:37.763417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e046d-b75d-46d2-a6b4-753f8e12f672 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:37.891914 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e31096-07a2-4200-819e-c00ec0f9e0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:37.892654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e31096-07a2-4200-819e-c00ec0f9e0cd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:37.892930 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e31096-07a2-4200-819e-c00ec0f9e0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:37.893214 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e31096-07a2-4200-819e-c00ec0f9e0cd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:37.905111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e31096-07a2-4200-819e-c00ec0f9e0cd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:37.906633 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e31096-07a2-4200-819e-c00ec0f9e0cd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:37.938334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a8e31096-07a2-4200-819e-c00ec0f9e0cd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 18:15:37.938334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:37.938334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:37.938334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:37.938334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:37.938334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:37.938334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:37.938334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:37.938334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:37.938334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:37.938334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:37.938334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:37.938334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:15:37.938334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:15:37.938334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:15:37.938334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:15:37.939356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:37.939356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:37.939356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:15:37.939356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:37.939356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:15:37.939356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:37.939356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:37.939356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:37.939356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:37.939356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:37.939356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:37.939356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:37.939356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:37.939356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:37.939356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:37.939356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:37.939356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:37.939356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:37.940271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:37.940271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:37.940271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:15:37.940271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:15:37.940271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:15:37.940271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:15:37.940271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:15:37.940271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:15:37.940271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 18:15:37.940271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:37.943514 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 308/622] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:15:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 18:15:37.950064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 315/623] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:15:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:15:37.952650 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacfcd8-8598-42d0-ab7d-97a33b107342 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:37.958599 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e31096-07a2-4200-819e-c00ec0f9e0cd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:37.977556 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:38.080077 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:38.139351 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacfcd8-8598-42d0-ab7d-97a33b107342 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:38.187817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e31096-07a2-4200-819e-c00ec0f9e0cd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:38.216474 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db629e85-4527-45d0-b710-46998f9d5590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:38.217289 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db629e85-4527-45d0-b710-46998f9d5590 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:38.217573 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db629e85-4527-45d0-b710-46998f9d5590 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:38.217877 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db629e85-4527-45d0-b710-46998f9d5590 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:38.234852 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db629e85-4527-45d0-b710-46998f9d5590 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:38.237709 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db629e85-4527-45d0-b710-46998f9d5590 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:38.241136 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900028b2-bffa-4171-8ffb-46563170811d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:38.245834 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900028b2-bffa-4171-8ffb-46563170811d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:38.245834 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900028b2-bffa-4171-8ffb-46563170811d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:38.245834 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900028b2-bffa-4171-8ffb-46563170811d None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:38.245834 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900028b2-bffa-4171-8ffb-46563170811d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:38.245834 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900028b2-bffa-4171-8ffb-46563170811d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:38.245834 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900028b2-bffa-4171-8ffb-46563170811d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:38.276437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-db629e85-4527-45d0-b710-46998f9d5590 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:38.276437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:38.276437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:38.276437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:38.276437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:38.276437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:38.276437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:38.276437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:38.276437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:38.276437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:38.276437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:38.276437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:38.276437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:38.276437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:38.276437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:38.278422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:38.278422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:38.278422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:38.278422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:38.278422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:38.278422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:38.278422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:38.278422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:38.278422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:38.278422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:38.278422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:38.278422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:38.278422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:38.278422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:38.278422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:38.278422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:38.278422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:38.279988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:38.279988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:38.279988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:38.279988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:38.280239 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 316/624] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Jun 30 18:15:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:38.280313 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 309/625] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:38.280313 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404f5ac-c50e-481a-9508-c098588e7d92 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:38.280313 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404f5ac-c50e-481a-9508-c098588e7d92 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:38.280313 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404f5ac-c50e-481a-9508-c098588e7d92 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:38.281618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 310/626] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:38.282893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404f5ac-c50e-481a-9508-c098588e7d92 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:38.283768 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae3dfe-0ea5-4efb-980d-3498fe7185ea None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:38.284105 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae3dfe-0ea5-4efb-980d-3498fe7185ea None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:38.284396 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae3dfe-0ea5-4efb-980d-3498fe7185ea None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:38.286495 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 317/627] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:38.290889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae3dfe-0ea5-4efb-980d-3498fe7185ea None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:38.377815 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:38.402285 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404f5ac-c50e-481a-9508-c098588e7d92 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:38.420902 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:38.423667 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae3dfe-0ea5-4efb-980d-3498fe7185ea None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:38.448145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe799d-e9cf-4a75-9209-cbc58d5cbf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:38.448860 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe799d-e9cf-4a75-9209-cbc58d5cbf70 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:38.449148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe799d-e9cf-4a75-9209-cbc58d5cbf70 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:38.449415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe799d-e9cf-4a75-9209-cbc58d5cbf70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:38.463394 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe799d-e9cf-4a75-9209-cbc58d5cbf70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:38.464003 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b06af-beb7-4601-93e8-1d542053b249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:38.464675 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b06af-beb7-4601-93e8-1d542053b249 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:38.464961 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b06af-beb7-4601-93e8-1d542053b249 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:38.465241 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b06af-beb7-4601-93e8-1d542053b249 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:38.466057 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54b06af-beb7-4601-93e8-1d542053b249 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:38.466300 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe799d-e9cf-4a75-9209-cbc58d5cbf70 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:38.466403 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54b06af-beb7-4601-93e8-1d542053b249 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:38.467675 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54b06af-beb7-4601-93e8-1d542053b249 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:38.485550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 318/628] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:38.487000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0115b-96f5-434a-93f7-6a5d3a5bde0b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:38.487308 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0115b-96f5-434a-93f7-6a5d3a5bde0b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:38.487584 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0115b-96f5-434a-93f7-6a5d3a5bde0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:38.503419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4cbe799d-e9cf-4a75-9209-cbc58d5cbf70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:38.503419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:38.503419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:38.503419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:38.503419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:38.503419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:38.503419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:38.503419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:38.503419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:38.503419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:38.503419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:38.503419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:38.503419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:38.503419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:38.503419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:38.504516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:38.504516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:38.504516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:38.504516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:38.504516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:38.504516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:38.504516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:38.504516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:38.504516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:38.504516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:38.504516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:38.504516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:38.504516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:38.504516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:38.504516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:38.504516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:38.504516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:38.505408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:38.505408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:38.505408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:38.505408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:38.506835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 311/629] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:38.508776 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1ea90-bfb7-4252-884d-c0f696875df6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:38.509290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1ea90-bfb7-4252-884d-c0f696875df6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:38.509818 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1ea90-bfb7-4252-884d-c0f696875df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:38.511686 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5fe0115b-96f5-434a-93f7-6a5d3a5bde0b None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:38.513056 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5fe0115b-96f5-434a-93f7-6a5d3a5bde0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:38.536901 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e9a1ea90-bfb7-4252-884d-c0f696875df6 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:38.539088 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e9a1ea90-bfb7-4252-884d-c0f696875df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:38.605968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 319/630] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:38.618218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0115b-96f5-434a-93f7-6a5d3a5bde0b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:38.643629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 312/631] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:38.654623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1ea90-bfb7-4252-884d-c0f696875df6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:38.695613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:38.780503 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0115b-96f5-434a-93f7-6a5d3a5bde0b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:38.817517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:38.826716 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1ea90-bfb7-4252-884d-c0f696875df6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:38.845935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b00d6-5f86-4046-9760-be92762d4491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:38.847332 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b00d6-5f86-4046-9760-be92762d4491 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:38.847853 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b00d6-5f86-4046-9760-be92762d4491 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:38.848358 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b00d6-5f86-4046-9760-be92762d4491 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:38.862394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b00d6-5f86-4046-9760-be92762d4491 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:38.865965 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b00d6-5f86-4046-9760-be92762d4491 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:38.869056 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79219ae6-7768-40b1-a69a-2867db1853d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:38.870020 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79219ae6-7768-40b1-a69a-2867db1853d7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:38.870314 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79219ae6-7768-40b1-a69a-2867db1853d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:38.870594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79219ae6-7768-40b1-a69a-2867db1853d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:38.871774 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79219ae6-7768-40b1-a69a-2867db1853d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:38.872091 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79219ae6-7768-40b1-a69a-2867db1853d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:38.874104 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79219ae6-7768-40b1-a69a-2867db1853d7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:38.888634 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 313/632] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:38.889428 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79219ae6-7768-40b1-a69a-2867db1853d7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:38.892901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-162b00d6-5f86-4046-9760-be92762d4491 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:38.892901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:38.892901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:38.892901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:38.892901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:38.892901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:38.892901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:38.892901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:38.892901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:38.892901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:38.892901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:38.892901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:38.892901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:15:38.892901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:15:38.892901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:15:38.895594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:15:38.895594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:38.895594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:38.895594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:38.895594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:38.895594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:38.895594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:38.895594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:38.895594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:38.895594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:38.895594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:38.895594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:38.895594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:38.895594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:38.895594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:38.895594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:38.895594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:38.897747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:15:38.897747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:15:38.897747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:15:38.897747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:15:38.897747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:15:38.897747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:15:38.897747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:38.897747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:38.897747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 320/633] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:38.897747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b00d6-5f86-4046-9760-be92762d4491 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:39.032358 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79219ae6-7768-40b1-a69a-2867db1853d7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:39.056857 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b00d6-5f86-4046-9760-be92762d4491 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:39.073466 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:39.082503 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96263b7-38ef-4311-b9ef-369fb12a5a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:39.083306 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96263b7-38ef-4311-b9ef-369fb12a5a0a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:39.083595 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96263b7-38ef-4311-b9ef-369fb12a5a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:39.084407 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96263b7-38ef-4311-b9ef-369fb12a5a0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:39.096724 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96263b7-38ef-4311-b9ef-369fb12a5a0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:39.097112 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd0aa3-66c5-40ba-ad0f-50a231e69bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:39.098087 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd0aa3-66c5-40ba-ad0f-50a231e69bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:39.098383 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd0aa3-66c5-40ba-ad0f-50a231e69bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:39.098665 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd0aa3-66c5-40ba-ad0f-50a231e69bc3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:39.105922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c96263b7-38ef-4311-b9ef-369fb12a5a0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:39.105922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:39.105922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:39.105922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:39.105922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:39.105922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:39.105922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:39.105922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:39.105922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:39.105922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:39.105922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:39.105922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:39.105922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:39.105922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:39.105922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:39.105922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:39.107952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:39.107952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:39.107952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:39.107952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:39.107952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:39.107952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:39.107952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:39.107952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:39.107952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:39.107952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:39.107952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:39.107952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:39.107952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:39.107952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:39.107952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:39.107952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:39.109005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:39.109005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:39.109005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:39.109005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:39.109005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:39.109005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:39.109005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:39.109005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:39.109005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:39.109005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:39.109005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:39.109005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:39.109005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 314/634] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 30 18:15:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:39.111201 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61164680-21ba-4757-a04c-82fe99788c4d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:39.111503 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61164680-21ba-4757-a04c-82fe99788c4d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:39.111906 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61164680-21ba-4757-a04c-82fe99788c4d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:39.113252 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 315/635] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:39.114033 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61164680-21ba-4757-a04c-82fe99788c4d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:39.115134 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd0aa3-66c5-40ba-ad0f-50a231e69bc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:39.124000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-63cd0aa3-66c5-40ba-ad0f-50a231e69bc3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:39.124000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:39.124000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:39.124000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:39.124000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:39.124000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:39.124000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:39.124000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:39.124000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:39.124000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:39.124000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:39.124000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:39.124000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:39.124000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:39.124000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:39.124000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:39.125143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:39.125143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:39.125143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:39.125143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:39.125143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:39.125143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:39.125143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:39.125143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:39.125143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:39.125143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:39.125143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:39.125143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:39.125143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:39.125143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:39.125143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:39.125143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:39.126032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:39.126032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:39.126032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:39.126032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:39.126032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:39.126032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:39.126032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:39.126032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:39.126032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:39.126032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:39.126032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:39.126032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:39.126032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 321/636] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 30 18:15:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:39.126743 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd0aa3-66c5-40ba-ad0f-50a231e69bc3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:39.144805 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:39.246457 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61164680-21ba-4757-a04c-82fe99788c4d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:39.251639 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd0aa3-66c5-40ba-ad0f-50a231e69bc3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:39.285221 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b90e84-0a5a-4cb8-aba9-a4505dc5a856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:39.285961 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b90e84-0a5a-4cb8-aba9-a4505dc5a856 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:39.286246 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b90e84-0a5a-4cb8-aba9-a4505dc5a856 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:39.286520 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b90e84-0a5a-4cb8-aba9-a4505dc5a856 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:39.291613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884037a-1636-401b-8f88-5e023b432c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:39.292277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884037a-1636-401b-8f88-5e023b432c17 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:39.292562 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884037a-1636-401b-8f88-5e023b432c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:39.292845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884037a-1636-401b-8f88-5e023b432c17 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:39.293693 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9884037a-1636-401b-8f88-5e023b432c17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:39.294006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9884037a-1636-401b-8f88-5e023b432c17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:39.294097 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b90e84-0a5a-4cb8-aba9-a4505dc5a856 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:39.295717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9884037a-1636-401b-8f88-5e023b432c17 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:39.295818 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b90e84-0a5a-4cb8-aba9-a4505dc5a856 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:39.310279 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 322/637] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:15:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:39.311287 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884037a-1636-401b-8f88-5e023b432c17 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:39.311486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e6b90e84-0a5a-4cb8-aba9-a4505dc5a856 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:39.311486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:39.311486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:39.311486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:39.311486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:39.311486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:39.311486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:39.311486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:39.311486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:39.311486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:39.311486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:39.311486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:39.311486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:15:39.311486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:15:39.311486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:15:39.312456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:15:39.312456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:39.312456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:39.312456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:39.312456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:39.312456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:39.312456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:39.312456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:39.312456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:39.312456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:39.312456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:39.312456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:39.312456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:39.312456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:39.312456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:39.312456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:39.312456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:39.313348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:15:39.313348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:15:39.313348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:15:39.313348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:15:39.313348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:15:39.313348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:15:39.313348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:39.313348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:39.313348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 316/638] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:39.314076 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31b84f-466c-4faa-882a-4a3a54157c30 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:39.314202 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31b84f-466c-4faa-882a-4a3a54157c30 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:39.314482 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31b84f-466c-4faa-882a-4a3a54157c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:39.331738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6a31b84f-466c-4faa-882a-4a3a54157c30 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:39.332933 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6a31b84f-466c-4faa-882a-4a3a54157c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:39.352947 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:39.414138 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 317/639] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:39.414965 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31b84f-466c-4faa-882a-4a3a54157c30 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:39.423711 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:39.444467 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884037a-1636-401b-8f88-5e023b432c17 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:39.480663 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bd140-0224-49e0-a218-ec57e3fccaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:39.481329 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bd140-0224-49e0-a218-ec57e3fccaba None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:39.481631 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bd140-0224-49e0-a218-ec57e3fccaba None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:39.481916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bd140-0224-49e0-a218-ec57e3fccaba None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:39.482662 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321bd140-0224-49e0-a218-ec57e3fccaba None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:39.482967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321bd140-0224-49e0-a218-ec57e3fccaba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:39.484341 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321bd140-0224-49e0-a218-ec57e3fccaba None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:39.492592 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 323/640] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 30 18:15:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:39.493416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bd140-0224-49e0-a218-ec57e3fccaba None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:39.536757 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31b84f-466c-4faa-882a-4a3a54157c30 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:39.572497 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1534cdb-0fe7-4e78-9ef4-b28a2f1a1f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:39.573169 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1534cdb-0fe7-4e78-9ef4-b28a2f1a1f88 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:39.573444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1534cdb-0fe7-4e78-9ef4-b28a2f1a1f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:39.573746 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1534cdb-0fe7-4e78-9ef4-b28a2f1a1f88 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:39.574487 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1534cdb-0fe7-4e78-9ef4-b28a2f1a1f88 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:39.574794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1534cdb-0fe7-4e78-9ef4-b28a2f1a1f88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:39.576171 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1534cdb-0fe7-4e78-9ef4-b28a2f1a1f88 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:39.585788 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 318/641] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 30 18:15:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:39.587089 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1534cdb-0fe7-4e78-9ef4-b28a2f1a1f88 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:39.593389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:39.600583 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bd140-0224-49e0-a218-ec57e3fccaba None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:39.635445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ee004-1053-49dc-9a0c-c97b3c878e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:39.636105 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ee004-1053-49dc-9a0c-c97b3c878e69 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:39.636392 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ee004-1053-49dc-9a0c-c97b3c878e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:39.636667 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ee004-1053-49dc-9a0c-c97b3c878e69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:39.637499 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ee004-1053-49dc-9a0c-c97b3c878e69 None admin] RBAC: Adding query filter params (role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d, scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b, user.id=b8751a4e92174d18b7b138392d2579a1) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:39.637842 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ee004-1053-49dc-9a0c-c97b3c878e69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:39.639302 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ee004-1053-49dc-9a0c-c97b3c878e69 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:39.645528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 324/642] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:15:39 2022] GET /identity/v3/role_assignments?user.id=b8751a4e92174d18b7b138392d2579a1&scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b&role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:39.647226 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ee004-1053-49dc-9a0c-c97b3c878e69 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:39.660449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:39.702557 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1534cdb-0fe7-4e78-9ef4-b28a2f1a1f88 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:39.745869 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b705857-a969-4b8f-9d40-771e094d29be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:39.747062 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b705857-a969-4b8f-9d40-771e094d29be None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:39.747637 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b705857-a969-4b8f-9d40-771e094d29be None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:39.747923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b705857-a969-4b8f-9d40-771e094d29be None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:39.748734 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b705857-a969-4b8f-9d40-771e094d29be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:39.749049 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b705857-a969-4b8f-9d40-771e094d29be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:39.750695 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b705857-a969-4b8f-9d40-771e094d29be None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:39.773179 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 319/643] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:15:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:39.773179 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b705857-a969-4b8f-9d40-771e094d29be None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:39.783453 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ee004-1053-49dc-9a0c-c97b3c878e69 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:39.827508 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf03532-becb-49f5-81e0-2b4320d9d747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:39.828342 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf03532-becb-49f5-81e0-2b4320d9d747 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:39.828639 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf03532-becb-49f5-81e0-2b4320d9d747 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:39.828927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf03532-becb-49f5-81e0-2b4320d9d747 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:39.841628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf03532-becb-49f5-81e0-2b4320d9d747 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:39.850092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-6bf03532-becb-49f5-81e0-2b4320d9d747 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:39.850092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:39.850092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:39.850092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:39.850092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:39.850092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:39.850092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:39.850092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:39.850092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:39.850092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:39.850092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:39.850092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:39.850092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:39.850092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:39.850092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:39.850092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:39.851159 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:39.851159 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:39.851159 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:39.851159 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:39.851159 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:39.851159 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:39.851159 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:39.851159 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:39.851159 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:39.851159 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:39.851159 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:39.851159 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:39.851159 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:39.851159 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:39.851159 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:39.851159 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:39.852030 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:39.852030 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:39.852030 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:39.852030 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:39.852030 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:39.852030 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:39.852030 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:39.852030 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:39.852030 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:39.852030 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:39.852030 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:39.852030 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:39.852030 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 325/644] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 30 18:15:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:39.852771 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf03532-becb-49f5-81e0-2b4320d9d747 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:39.877411 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:39.909715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b705857-a969-4b8f-9d40-771e094d29be None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:39.939118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:39.952436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4e171-35c4-422e-ae87-e44898456055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:39.952436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4e171-35c4-422e-ae87-e44898456055 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:39.952436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4e171-35c4-422e-ae87-e44898456055 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:39.952436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4e171-35c4-422e-ae87-e44898456055 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:39.965703 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4e171-35c4-422e-ae87-e44898456055 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:39.967178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4e171-35c4-422e-ae87-e44898456055 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:39.969612 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf03532-becb-49f5-81e0-2b4320d9d747 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:39.985674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-34c4e171-35c4-422e-ae87-e44898456055 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:39.985674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:39.985674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:39.985674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:39.985674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:39.985674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:39.985674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:39.985674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:39.985674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:39.985674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:39.985674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:39.985674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:39.985674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:39.985674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:39.985674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:39.986597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:39.986597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:39.986597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:39.986597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:39.986597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:39.986597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:39.986597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:39.986597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:39.986597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:39.986597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:39.986597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:39.986597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:39.986597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:39.986597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:39.986597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:39.986597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:39.986597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:39.992279 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:39.992279 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:39.992279 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:39.992279 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:39.992279 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 320/645] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:39.992279 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4e171-35c4-422e-ae87-e44898456055 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:40.033081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74662b-8adf-4b98-bd12-4f11ab1e0318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:40.034178 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74662b-8adf-4b98-bd12-4f11ab1e0318 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:40.034645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74662b-8adf-4b98-bd12-4f11ab1e0318 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:40.034923 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74662b-8adf-4b98-bd12-4f11ab1e0318 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:40.047923 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c74662b-8adf-4b98-bd12-4f11ab1e0318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:40.049386 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c74662b-8adf-4b98-bd12-4f11ab1e0318 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:40.071645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-7c74662b-8adf-4b98-bd12-4f11ab1e0318 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:40.071645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:40.071645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:40.071645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:40.071645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:40.071645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:40.071645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:40.071645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:40.071645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:40.071645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:40.071645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:40.071645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:40.071645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:40.071645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:40.071645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:40.073013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:40.073013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:40.073013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:40.073013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:40.073013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:40.073013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:40.073013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:40.073013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:40.073013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:40.073013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:40.073013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:40.073013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:40.073013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:40.073013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:40.073013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:40.073013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:40.073013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:40.075113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:40.075113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:40.075113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:40.075113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:40.075113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 326/646] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:40.078418 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74662b-8adf-4b98-bd12-4f11ab1e0318 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:40.134568 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4e171-35c4-422e-ae87-e44898456055 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:40.160940 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:40.178590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1cdde3-a840-4537-b070-345f9a162ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:40.179722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1cdde3-a840-4537-b070-345f9a162ffa None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:40.179996 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1cdde3-a840-4537-b070-345f9a162ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:40.180749 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1cdde3-a840-4537-b070-345f9a162ffa None admin] PATH_INFO: `/v3/projects/8c7e1ee12aa14b7ab13e07822826b25b/users/4bd521b9e93f4b3b8af64d46f36c019e/roles/1ccfbd20ff5f4d068de0fe3bf93ebe6d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:40.215599 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1cdde3-a840-4537-b070-345f9a162ffa None admin] RBAC: Authorizing `identity:create_grant(project_id=8c7e1ee12aa14b7ab13e07822826b25b, user_id=4bd521b9e93f4b3b8af64d46f36c019e, role_id=1ccfbd20ff5f4d068de0fe3bf93ebe6d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:40.216808 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74662b-8adf-4b98-bd12-4f11ab1e0318 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:40.217350 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1cdde3-a840-4537-b070-345f9a162ffa None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:40.232013 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 321/647] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:15:39 2022] PUT /identity/v3/projects/8c7e1ee12aa14b7ab13e07822826b25b/users/4bd521b9e93f4b3b8af64d46f36c019e/roles/1ccfbd20ff5f4d068de0fe3bf93ebe6d => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:15:40.233303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1cdde3-a840-4537-b070-345f9a162ffa None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:40.273867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:40.274085 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af199c-9e15-4d7d-8e5c-5ce63cd408c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:40.275070 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af199c-9e15-4d7d-8e5c-5ce63cd408c2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:40.275070 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af199c-9e15-4d7d-8e5c-5ce63cd408c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:40.275366 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af199c-9e15-4d7d-8e5c-5ce63cd408c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:40.276140 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af199c-9e15-4d7d-8e5c-5ce63cd408c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:40.276492 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af199c-9e15-4d7d-8e5c-5ce63cd408c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:40.281915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af199c-9e15-4d7d-8e5c-5ce63cd408c2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:40.297072 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 327/648] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 30 18:15:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:40.298418 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af199c-9e15-4d7d-8e5c-5ce63cd408c2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:40.384848 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1cdde3-a840-4537-b070-345f9a162ffa None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:40.430634 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269f4fe-aba9-46ed-bc63-2b2f50de5475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:40.431283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269f4fe-aba9-46ed-bc63-2b2f50de5475 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:40.431567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269f4fe-aba9-46ed-bc63-2b2f50de5475 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:40.431840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269f4fe-aba9-46ed-bc63-2b2f50de5475 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:40.432884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269f4fe-aba9-46ed-bc63-2b2f50de5475 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:40.433325 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269f4fe-aba9-46ed-bc63-2b2f50de5475 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:40.435098 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269f4fe-aba9-46ed-bc63-2b2f50de5475 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:40.435991 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af199c-9e15-4d7d-8e5c-5ce63cd408c2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:40.451818 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 322/649] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:40.453013 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269f4fe-aba9-46ed-bc63-2b2f50de5475 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:40.491175 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252348dd-9195-46a2-906a-db1124f74c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:40.492047 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252348dd-9195-46a2-906a-db1124f74c80 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:40.492360 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252348dd-9195-46a2-906a-db1124f74c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:40.492645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252348dd-9195-46a2-906a-db1124f74c80 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:40.493384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252348dd-9195-46a2-906a-db1124f74c80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:40.493753 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252348dd-9195-46a2-906a-db1124f74c80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:40.495162 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252348dd-9195-46a2-906a-db1124f74c80 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:40.496938 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:40.510206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 328/650] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:40.511094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252348dd-9195-46a2-906a-db1124f74c80 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:40.591391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:40.604235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269f4fe-aba9-46ed-bc63-2b2f50de5475 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:40.644731 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee3be3-6d94-4be8-b56b-250de82b80af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:40.645592 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee3be3-6d94-4be8-b56b-250de82b80af None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:40.645893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee3be3-6d94-4be8-b56b-250de82b80af None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:40.646174 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee3be3-6d94-4be8-b56b-250de82b80af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:40.660732 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee3be3-6d94-4be8-b56b-250de82b80af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:40.662998 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee3be3-6d94-4be8-b56b-250de82b80af None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:40.682374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e4ee3be3-6d94-4be8-b56b-250de82b80af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:40.682374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:40.682374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:40.682374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:40.682374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:40.682374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:40.682374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:40.682374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:40.682374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:40.682374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:40.682374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:40.682374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:40.682374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:40.682374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:40.682374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:40.683333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:40.683333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:40.683333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:40.683333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:40.683333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:40.683333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:40.683333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:40.683333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:40.683333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:40.683333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:40.683333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:40.683333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:40.683333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:40.683333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:40.683333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:40.683333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:40.683333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:40.684666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:40.684666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:40.684666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:40.684666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:40.685115 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252348dd-9195-46a2-906a-db1124f74c80 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:40.685194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 323/651] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:40.685194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee3be3-6d94-4be8-b56b-250de82b80af None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:40.729845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38cc57-2d71-4f97-84b9-4d2d913109e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:40.730685 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38cc57-2d71-4f97-84b9-4d2d913109e5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:40.730970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38cc57-2d71-4f97-84b9-4d2d913109e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:40.731250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38cc57-2d71-4f97-84b9-4d2d913109e5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:40.739955 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38cc57-2d71-4f97-84b9-4d2d913109e5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:40.741691 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38cc57-2d71-4f97-84b9-4d2d913109e5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:40.758445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ab38cc57-2d71-4f97-84b9-4d2d913109e5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 18:15:40.758445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:40.758445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:40.758445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:40.758445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:40.758445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:40.758445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:40.758445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:40.758445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:40.758445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:40.758445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:40.758445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:40.758445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:15:40.758445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:15:40.758445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:15:40.758445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:15:40.760778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:40.760778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:40.760778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:15:40.760778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:40.760778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:15:40.760778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:40.760778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:40.760778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:40.760778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:40.760778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:40.760778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:40.760778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:40.760778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:40.760778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:40.760778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:40.760778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:40.760778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:40.760778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:40.763044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:40.763044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:40.763044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:15:40.763044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:15:40.763044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:15:40.763044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:15:40.763044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:15:40.763044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:15:40.763044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 18:15:40.763044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:40.770085 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 329/652] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:15:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:40.770085 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38cc57-2d71-4f97-84b9-4d2d913109e5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:40.800940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee3be3-6d94-4be8-b56b-250de82b80af None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:40.834588 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:40.839616 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590a97f-c703-497d-99ab-4779e8af755f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:40.840425 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590a97f-c703-497d-99ab-4779e8af755f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:40.840711 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590a97f-c703-497d-99ab-4779e8af755f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:40.840984 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590a97f-c703-497d-99ab-4779e8af755f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:40.858787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590a97f-c703-497d-99ab-4779e8af755f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:40.860652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590a97f-c703-497d-99ab-4779e8af755f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:40.884893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4590a97f-c703-497d-99ab-4779e8af755f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 18:15:40.884893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:40.884893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:40.884893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:40.884893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:40.884893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:40.884893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:40.884893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:40.884893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:40.884893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:40.884893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:40.884893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:40.884893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:15:40.884893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:15:40.884893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:15:40.884893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:15:40.886040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:40.886040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:40.886040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:15:40.886040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:40.886040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:15:40.886040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:40.886040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:40.886040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:40.886040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:40.886040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:40.886040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:40.886040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:40.886040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:40.886040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:40.886040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:40.886040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:40.886040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:40.886040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:40.886983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:40.886983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:40.886983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:15:40.886983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:15:40.886983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:15:40.886983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:15:40.886983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:15:40.886983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:15:40.886983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 18:15:40.886983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:40.889764 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 324/653] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:15:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:40.891034 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590a97f-c703-497d-99ab-4779e8af755f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:40.895756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:40.904255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38cc57-2d71-4f97-84b9-4d2d913109e5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:40.944678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7147c-4dbe-47d5-b90c-9e89c07a97d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:40.945367 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7147c-4dbe-47d5-b90c-9e89c07a97d0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:40.945843 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7147c-4dbe-47d5-b90c-9e89c07a97d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:40.946041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7147c-4dbe-47d5-b90c-9e89c07a97d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:40.946952 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7147c-4dbe-47d5-b90c-9e89c07a97d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:40.947264 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7147c-4dbe-47d5-b90c-9e89c07a97d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:40.948714 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7147c-4dbe-47d5-b90c-9e89c07a97d0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:40.968559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 330/654] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:40.969726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7147c-4dbe-47d5-b90c-9e89c07a97d0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:41.032019 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590a97f-c703-497d-99ab-4779e8af755f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:41.074640 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36e6c8-158f-42a4-aeab-c4b8556bb4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:41.075711 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36e6c8-158f-42a4-aeab-c4b8556bb4fa None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:41.076201 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36e6c8-158f-42a4-aeab-c4b8556bb4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:41.076679 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36e6c8-158f-42a4-aeab-c4b8556bb4fa None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:41.077786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f36e6c8-158f-42a4-aeab-c4b8556bb4fa None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:41.078321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f36e6c8-158f-42a4-aeab-c4b8556bb4fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:41.080310 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f36e6c8-158f-42a4-aeab-c4b8556bb4fa None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:41.096949 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7147c-4dbe-47d5-b90c-9e89c07a97d0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:41.105260 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 325/655] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Jun 30 18:15:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:41.106489 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36e6c8-158f-42a4-aeab-c4b8556bb4fa None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:41.112713 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:41.137343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc522b-b38e-43c2-bf0d-91cc5e4561c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:41.138143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc522b-b38e-43c2-bf0d-91cc5e4561c0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:41.138431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc522b-b38e-43c2-bf0d-91cc5e4561c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:41.138710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc522b-b38e-43c2-bf0d-91cc5e4561c0 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:41.139547 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc522b-b38e-43c2-bf0d-91cc5e4561c0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:41.139929 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc522b-b38e-43c2-bf0d-91cc5e4561c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:41.141378 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc522b-b38e-43c2-bf0d-91cc5e4561c0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:41.165233 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 331/656] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Jun 30 18:15:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:41.165904 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc522b-b38e-43c2-bf0d-91cc5e4561c0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:41.172582 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:41.215442 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36e6c8-158f-42a4-aeab-c4b8556bb4fa None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:41.254098 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336319fb-bd3d-4319-b362-0515625cc6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:41.254886 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336319fb-bd3d-4319-b362-0515625cc6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:41.255170 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336319fb-bd3d-4319-b362-0515625cc6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:41.255451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336319fb-bd3d-4319-b362-0515625cc6b8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:41.268339 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336319fb-bd3d-4319-b362-0515625cc6b8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:41.271023 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336319fb-bd3d-4319-b362-0515625cc6b8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:41.282953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc522b-b38e-43c2-bf0d-91cc5e4561c0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:41.291722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-336319fb-bd3d-4319-b362-0515625cc6b8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 18:15:41.291722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:41.291722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:41.291722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:41.291722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:41.291722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:41.291722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:41.291722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:41.291722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:41.291722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:41.291722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:41.291722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:41.291722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:15:41.291722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:15:41.291722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:15:41.291722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:15:41.292660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:41.292660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:41.292660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:15:41.292660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:41.292660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:15:41.292660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:41.292660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:41.292660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:41.292660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:41.292660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:41.292660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:41.292660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:41.292660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:41.292660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:41.292660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:41.292660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:41.292660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:41.292660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:41.293599 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:41.293599 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:41.293599 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:15:41.293599 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:15:41.293599 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:15:41.293599 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:15:41.293599 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:15:41.293599 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:15:41.293599 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 18:15:41.293599 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:41.300261 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 326/657] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 30 18:15:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:41.300261 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336319fb-bd3d-4319-b362-0515625cc6b8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:41.327994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332046a-b49e-4899-8eeb-3caf55b075c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:41.329376 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332046a-b49e-4899-8eeb-3caf55b075c4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:41.329905 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332046a-b49e-4899-8eeb-3caf55b075c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:41.330388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332046a-b49e-4899-8eeb-3caf55b075c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:41.388598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:41.425554 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b332046a-b49e-4899-8eeb-3caf55b075c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:41.426922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336319fb-bd3d-4319-b362-0515625cc6b8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:41.427833 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b332046a-b49e-4899-8eeb-3caf55b075c4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:41.453623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:41.469470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f04fa-e39f-4694-8d8d-07e2275aed91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:41.470177 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f04fa-e39f-4694-8d8d-07e2275aed91 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:41.470465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f04fa-e39f-4694-8d8d-07e2275aed91 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:41.470751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f04fa-e39f-4694-8d8d-07e2275aed91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:41.549901 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f04fa-e39f-4694-8d8d-07e2275aed91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:41.551425 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f04fa-e39f-4694-8d8d-07e2275aed91 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:41.562378 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 332/658] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:15:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:15:41.563673 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332046a-b49e-4899-8eeb-3caf55b075c4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:41.641289 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:41.676363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 327/659] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:15:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:15:41.676363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f04fa-e39f-4694-8d8d-07e2275aed91 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:41.686402 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332046a-b49e-4899-8eeb-3caf55b075c4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:41.710008 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:41.730072 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a458ed-ba92-411d-80d8-bf173f25ec79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:41.731132 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a458ed-ba92-411d-80d8-bf173f25ec79 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:41.731617 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a458ed-ba92-411d-80d8-bf173f25ec79 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:41.732093 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a458ed-ba92-411d-80d8-bf173f25ec79 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:41.733139 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a458ed-ba92-411d-80d8-bf173f25ec79 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:41.733700 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a458ed-ba92-411d-80d8-bf173f25ec79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:41.735692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a458ed-ba92-411d-80d8-bf173f25ec79 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:41.761043 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 333/660] 10.222.0.33 () {56 vars in 1139 bytes} [Thu Jun 30 18:15:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:41.762497 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a458ed-ba92-411d-80d8-bf173f25ec79 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:41.779080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f04fa-e39f-4694-8d8d-07e2275aed91 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:41.815276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac9f0f-7074-4534-9885-8de82cb5bcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:41.816074 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac9f0f-7074-4534-9885-8de82cb5bcfb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:41.816362 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac9f0f-7074-4534-9885-8de82cb5bcfb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:41.816645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac9f0f-7074-4534-9885-8de82cb5bcfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:41.827849 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac9f0f-7074-4534-9885-8de82cb5bcfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:41.829326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac9f0f-7074-4534-9885-8de82cb5bcfb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:41.847262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-5aac9f0f-7074-4534-9885-8de82cb5bcfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:41.847262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:41.847262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:41.847262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:41.847262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:41.847262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:41.847262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:41.847262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:41.847262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:41.847262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:41.847262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:41.847262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:41.847262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:41.847262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:41.847262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:41.848215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:41.848215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:41.848215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:41.848215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:41.848215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:41.848215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:41.848215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:41.848215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:41.848215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:41.848215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:41.848215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:41.848215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:41.848215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:41.848215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:41.848215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:41.848215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:41.848215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:41.849095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:41.849095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:41.849095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:41.849095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:41.849095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 328/661] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:41.850207 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac9f0f-7074-4534-9885-8de82cb5bcfb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:41.875383 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a458ed-ba92-411d-80d8-bf173f25ec79 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:41.911460 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:41.919903 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6eac4a-ce4e-401b-9b44-48f97b4bcbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:41.921086 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6eac4a-ce4e-401b-9b44-48f97b4bcbc5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:41.921697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6eac4a-ce4e-401b-9b44-48f97b4bcbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:41.922158 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6eac4a-ce4e-401b-9b44-48f97b4bcbc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:41.936259 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6eac4a-ce4e-401b-9b44-48f97b4bcbc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:41.937521 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:41.938234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6eac4a-ce4e-401b-9b44-48f97b4bcbc5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:41.959472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ee6eac4a-ce4e-401b-9b44-48f97b4bcbc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:41.959472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:41.959472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:41.959472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:41.959472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:41.959472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:41.959472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:41.959472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:41.959472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:41.959472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:41.959472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:41.959472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:41.959472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:41.959472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:41.959472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:41.960462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:41.960462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:41.960462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:41.960462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:41.960462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:41.960462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:41.960462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:41.960462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:41.960462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:41.960462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:41.960462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:41.960462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:41.960462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:41.960462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:41.960462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:41.960462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:41.960462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:41.961364 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:41.961364 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:41.961364 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:41.961364 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:41.962219 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 334/662] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:41.964166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6eac4a-ce4e-401b-9b44-48f97b4bcbc5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:41.981532 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac9f0f-7074-4534-9885-8de82cb5bcfb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:42.019327 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b76c8-81f6-4c6f-a325-6254474cfb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:42.019983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b76c8-81f6-4c6f-a325-6254474cfb6b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:42.020272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b76c8-81f6-4c6f-a325-6254474cfb6b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:42.020554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b76c8-81f6-4c6f-a325-6254474cfb6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:42.092766 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6eac4a-ce4e-401b-9b44-48f97b4bcbc5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:42.096584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b76c8-81f6-4c6f-a325-6254474cfb6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:42.098093 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b76c8-81f6-4c6f-a325-6254474cfb6b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:42.128811 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d75d7-2090-4225-9d56-631275f172d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:42.129893 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d75d7-2090-4225-9d56-631275f172d3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:42.130174 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d75d7-2090-4225-9d56-631275f172d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:42.130451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d75d7-2090-4225-9d56-631275f172d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:42.131721 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584d75d7-2090-4225-9d56-631275f172d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:42.132113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584d75d7-2090-4225-9d56-631275f172d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:42.133503 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584d75d7-2090-4225-9d56-631275f172d3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:42.145516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 335/663] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:42.147979 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4145df-d821-4494-8fed-e7d36b2c62cd None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:42.148276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4145df-d821-4494-8fed-e7d36b2c62cd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:42.148553 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4145df-d821-4494-8fed-e7d36b2c62cd None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:42.153616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 336/664] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:42.153616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4145df-d821-4494-8fed-e7d36b2c62cd None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:42.181753 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:42.202958 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:42.230335 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 329/665] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:15:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 18:15:42.231092 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b76c8-81f6-4c6f-a325-6254474cfb6b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:42.285844 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4145df-d821-4494-8fed-e7d36b2c62cd None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:42.326697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ebf14-a305-46db-8f60-a9e1e84b3848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:42.327368 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ebf14-a305-46db-8f60-a9e1e84b3848 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:42.327654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ebf14-a305-46db-8f60-a9e1e84b3848 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:42.327932 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ebf14-a305-46db-8f60-a9e1e84b3848 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:42.328669 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624ebf14-a305-46db-8f60-a9e1e84b3848 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:42.328973 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624ebf14-a305-46db-8f60-a9e1e84b3848 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:42.330481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624ebf14-a305-46db-8f60-a9e1e84b3848 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:42.344546 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 337/666] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:42.350933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fada871-2aa5-462f-b2d2-c0438c4e3053 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:42.351427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fada871-2aa5-462f-b2d2-c0438c4e3053 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:42.351639 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fada871-2aa5-462f-b2d2-c0438c4e3053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:42.355930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b76c8-81f6-4c6f-a325-6254474cfb6b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:42.368544 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3fada871-2aa5-462f-b2d2-c0438c4e3053 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:42.369792 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3fada871-2aa5-462f-b2d2-c0438c4e3053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:42.403306 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480d11b-912e-4c52-8427-14d166f8b8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:42.404453 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480d11b-912e-4c52-8427-14d166f8b8b6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:42.404736 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480d11b-912e-4c52-8427-14d166f8b8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:42.405012 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480d11b-912e-4c52-8427-14d166f8b8b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:42.414479 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480d11b-912e-4c52-8427-14d166f8b8b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:42.414757 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480d11b-912e-4c52-8427-14d166f8b8b6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:42.430541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3480d11b-912e-4c52-8427-14d166f8b8b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:42.430541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:42.430541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:42.430541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:42.430541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:42.430541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:42.430541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:42.430541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:42.430541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:42.430541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:42.430541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:42.430541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:42.430541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:15:42.430541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:15:42.430541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:15:42.431527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:15:42.431527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:42.431527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:42.431527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:42.431527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:42.431527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:42.431527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:42.431527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:42.431527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:42.431527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:42.431527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:42.431527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:42.431527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:42.431527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:42.431527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:42.431527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:42.431527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:42.432426 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:15:42.432426 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:15:42.432426 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:15:42.432426 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:15:42.432426 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:15:42.432426 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:15:42.432426 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:42.432426 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:42.432426 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 330/667] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:42.432900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480d11b-912e-4c52-8427-14d166f8b8b6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:42.473783 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 338/668] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:42.473783 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fada871-2aa5-462f-b2d2-c0438c4e3053 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:42.477884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:42.505190 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:42.564509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480d11b-912e-4c52-8427-14d166f8b8b6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:42.595291 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fada871-2aa5-462f-b2d2-c0438c4e3053 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:42.613758 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f60d07-f58c-400d-a3c1-cec5fd979cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:42.614557 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f60d07-f58c-400d-a3c1-cec5fd979cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:42.614846 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f60d07-f58c-400d-a3c1-cec5fd979cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:42.615305 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f60d07-f58c-400d-a3c1-cec5fd979cb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:42.628749 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f60d07-f58c-400d-a3c1-cec5fd979cb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:42.630287 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f60d07-f58c-400d-a3c1-cec5fd979cb0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:42.638638 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19adf14-3ab6-4ce7-bb10-25bdeff6e1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:42.639442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19adf14-3ab6-4ce7-bb10-25bdeff6e1bf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:42.639726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19adf14-3ab6-4ce7-bb10-25bdeff6e1bf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:42.640006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19adf14-3ab6-4ce7-bb10-25bdeff6e1bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:42.650461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-59f60d07-f58c-400d-a3c1-cec5fd979cb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:42.650461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:42.650461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:42.650461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:42.650461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:42.650461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:42.650461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:42.650461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:42.650461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:42.650461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:42.650461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:42.650461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:42.650461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:42.650461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:42.650461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:42.652051 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:42.652051 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:42.652051 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:42.652051 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:42.652051 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:42.652051 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:42.652051 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:42.652051 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:42.652051 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:42.652051 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:42.652051 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:42.652051 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:42.652051 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:42.652051 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:42.652051 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:42.652051 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:42.652051 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:42.653624 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:42.653624 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:42.653624 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:42.653624 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:42.653624 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 331/669] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:42.654087 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19adf14-3ab6-4ce7-bb10-25bdeff6e1bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:42.654163 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f60d07-f58c-400d-a3c1-cec5fd979cb0 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:42.654232 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19adf14-3ab6-4ce7-bb10-25bdeff6e1bf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:42.670908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a19adf14-3ab6-4ce7-bb10-25bdeff6e1bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:42.670908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:42.670908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:42.670908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:42.670908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:42.670908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:42.670908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:42.670908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:42.670908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:42.670908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:42.670908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:42.670908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:42.670908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:15:42.670908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:15:42.670908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:15:42.671953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:15:42.671953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:42.671953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:42.671953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:42.671953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:42.671953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:42.671953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:42.671953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:42.671953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:42.671953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:42.671953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:42.671953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:42.671953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:42.671953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:42.671953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:42.671953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:42.671953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:42.673041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:15:42.673041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:15:42.673041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:15:42.673041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:15:42.673041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:15:42.673041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:15:42.673041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:42.673041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:42.673041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 339/670] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:42.673548 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ba803-ab1e-42ca-bea8-2f24d678da8a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:42.673978 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ba803-ab1e-42ca-bea8-2f24d678da8a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:42.674079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ba803-ab1e-42ca-bea8-2f24d678da8a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:42.677854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 340/671] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:42.679499 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ba803-ab1e-42ca-bea8-2f24d678da8a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:42.758360 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:42.776447 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f60d07-f58c-400d-a3c1-cec5fd979cb0 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:42.784777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:42.796932 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ba803-ab1e-42ca-bea8-2f24d678da8a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:42.823408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bec62-81c4-487a-b4cd-b73cb4d3c6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:42.824456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bec62-81c4-487a-b4cd-b73cb4d3c6fb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:42.824917 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bec62-81c4-487a-b4cd-b73cb4d3c6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:42.825379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bec62-81c4-487a-b4cd-b73cb4d3c6fb None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:42.826625 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8bec62-81c4-487a-b4cd-b73cb4d3c6fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:42.827143 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8bec62-81c4-487a-b4cd-b73cb4d3c6fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:42.829802 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8bec62-81c4-487a-b4cd-b73cb4d3c6fb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:42.850887 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 332/672] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:15:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:42.851664 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bec62-81c4-487a-b4cd-b73cb4d3c6fb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:42.859261 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef03c38-3045-4b15-b46c-48bbab99e1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:42.860472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef03c38-3045-4b15-b46c-48bbab99e1e3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:42.860940 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef03c38-3045-4b15-b46c-48bbab99e1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:42.861350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef03c38-3045-4b15-b46c-48bbab99e1e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:42.878894 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef03c38-3045-4b15-b46c-48bbab99e1e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:42.893394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-cef03c38-3045-4b15-b46c-48bbab99e1e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:15:42.893394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:42.893394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:42.893394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:42.893394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:42.893394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:42.893394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:42.893394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:42.893394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:42.893394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:42.893394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:42.893394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:42.893394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:42.893394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:42.893394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:42.893394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:42.894559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:42.894559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:42.894559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:42.894559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:42.894559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:42.894559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:42.894559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:42.894559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:42.894559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:42.894559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:42.894559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:42.894559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:42.894559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:42.894559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:42.894559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:42.894559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:42.895384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:42.895384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:42.895384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:42.895384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:42.895384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:42.895384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:42.895384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:42.895384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:42.895384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:42.895384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:42.895384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:15:42.895384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:42.895384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 341/673] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:15:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:42.896078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef03c38-3045-4b15-b46c-48bbab99e1e3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:42.987138 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bec62-81c4-487a-b4cd-b73cb4d3c6fb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:43.024618 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef03c38-3045-4b15-b46c-48bbab99e1e3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:43.025894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee67540-ac91-4069-9cd0-109a0d37258e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:43.026465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee67540-ac91-4069-9cd0-109a0d37258e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:43.027424 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee67540-ac91-4069-9cd0-109a0d37258e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:43.027424 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee67540-ac91-4069-9cd0-109a0d37258e None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:43.027729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee67540-ac91-4069-9cd0-109a0d37258e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:43.028037 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee67540-ac91-4069-9cd0-109a0d37258e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:43.029440 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee67540-ac91-4069-9cd0-109a0d37258e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:43.045199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 333/674] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:43.047104 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473551b-203e-454f-a1ea-67a8e1fb02da None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:43.047437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473551b-203e-454f-a1ea-67a8e1fb02da None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:43.047851 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473551b-203e-454f-a1ea-67a8e1fb02da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:43.056222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:43.070353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0739ad-787a-44a3-997f-55c2f1d65873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:43.070637 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b473551b-203e-454f-a1ea-67a8e1fb02da None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:43.071015 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0739ad-787a-44a3-997f-55c2f1d65873 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:43.071297 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0739ad-787a-44a3-997f-55c2f1d65873 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:43.071572 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0739ad-787a-44a3-997f-55c2f1d65873 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:43.071722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b473551b-203e-454f-a1ea-67a8e1fb02da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:43.072364 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0739ad-787a-44a3-997f-55c2f1d65873 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:43.072671 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0739ad-787a-44a3-997f-55c2f1d65873 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:43.074164 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0739ad-787a-44a3-997f-55c2f1d65873 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:43.092096 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 342/675] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:15:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:43.093548 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0739ad-787a-44a3-997f-55c2f1d65873 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:43.113904 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:43.165275 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 334/676] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:43.166280 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473551b-203e-454f-a1ea-67a8e1fb02da None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:43.219961 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0739ad-787a-44a3-997f-55c2f1d65873 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:43.257655 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2ee2a-05f0-4e15-8e92-4afc7ed2be2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:43.258282 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2ee2a-05f0-4e15-8e92-4afc7ed2be2b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:43.258566 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2ee2a-05f0-4e15-8e92-4afc7ed2be2b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:43.258849 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2ee2a-05f0-4e15-8e92-4afc7ed2be2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:43.259710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2ee2a-05f0-4e15-8e92-4afc7ed2be2b None admin] RBAC: Adding query filter params (role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d, scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b, user.id=37a9fea702cd405ea9fc14ec2c689648) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:43.260023 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2ee2a-05f0-4e15-8e92-4afc7ed2be2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:43.261493 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2ee2a-05f0-4e15-8e92-4afc7ed2be2b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:43.268140 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 343/677] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:15:43 2022] GET /identity/v3/role_assignments?user.id=37a9fea702cd405ea9fc14ec2c689648&scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b&role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:43.269055 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2ee2a-05f0-4e15-8e92-4afc7ed2be2b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:43.299067 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473551b-203e-454f-a1ea-67a8e1fb02da None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:43.355557 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c729ca-3aa9-4f83-ab0d-8232dc303cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:43.356797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c729ca-3aa9-4f83-ab0d-8232dc303cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:43.357383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c729ca-3aa9-4f83-ab0d-8232dc303cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:43.357846 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c729ca-3aa9-4f83-ab0d-8232dc303cb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:43.359105 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c729ca-3aa9-4f83-ab0d-8232dc303cb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:43.359463 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c729ca-3aa9-4f83-ab0d-8232dc303cb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:43.360963 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c729ca-3aa9-4f83-ab0d-8232dc303cb2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:43.363378 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:43.375531 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 335/678] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:15:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:43.376347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c729ca-3aa9-4f83-ab0d-8232dc303cb2 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:43.386094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:43.418621 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2ee2a-05f0-4e15-8e92-4afc7ed2be2b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:43.465002 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546fc169-091e-4fb3-92c9-b1daa624cc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:43.466744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546fc169-091e-4fb3-92c9-b1daa624cc7b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:43.467108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546fc169-091e-4fb3-92c9-b1daa624cc7b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:43.467407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546fc169-091e-4fb3-92c9-b1daa624cc7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:43.475911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546fc169-091e-4fb3-92c9-b1daa624cc7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:43.477394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546fc169-091e-4fb3-92c9-b1daa624cc7b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:43.493687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-546fc169-091e-4fb3-92c9-b1daa624cc7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:43.493687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:43.493687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:43.493687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:43.493687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:43.493687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:43.493687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:43.493687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:43.493687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:43.493687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:43.493687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:43.493687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:43.493687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:15:43.493687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:15:43.493687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:15:43.495113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:15:43.495113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:43.495113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:43.495113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:43.495113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:43.495113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:43.495113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:43.495113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:43.495113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:43.495113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:43.495113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:43.495113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:43.495113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:43.495113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:43.495113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:43.495113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:43.495113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:43.496371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:15:43.496371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:15:43.496371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:15:43.496371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:15:43.496371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:15:43.496371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:15:43.496371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:43.496371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:43.496371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 344/679] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:43.496371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546fc169-091e-4fb3-92c9-b1daa624cc7b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:43.534110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c729ca-3aa9-4f83-ab0d-8232dc303cb2 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:43.587026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9295497-1a8a-4d44-a3f9-d5dd5bf62ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:43.587589 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9295497-1a8a-4d44-a3f9-d5dd5bf62ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:43.587878 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9295497-1a8a-4d44-a3f9-d5dd5bf62ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:43.588154 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9295497-1a8a-4d44-a3f9-d5dd5bf62ad1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:43.589021 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9295497-1a8a-4d44-a3f9-d5dd5bf62ad1 None admin] RBAC: Adding query filter params (role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d, scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b, user.id=92c70ce830ea4dd981716d622b16d595) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:43.589320 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9295497-1a8a-4d44-a3f9-d5dd5bf62ad1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:43.590855 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9295497-1a8a-4d44-a3f9-d5dd5bf62ad1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:43.598385 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 336/680] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:15:43 2022] GET /identity/v3/role_assignments?user.id=92c70ce830ea4dd981716d622b16d595&scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b&role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:43.599232 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9295497-1a8a-4d44-a3f9-d5dd5bf62ad1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:43.652691 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546fc169-091e-4fb3-92c9-b1daa624cc7b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:43.698827 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8691a03-666b-4a5b-9adb-eef0d1901c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:43.700287 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8691a03-666b-4a5b-9adb-eef0d1901c39 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:43.700798 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8691a03-666b-4a5b-9adb-eef0d1901c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:43.701381 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8691a03-666b-4a5b-9adb-eef0d1901c39 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:43.708073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:43.721417 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:43.730872 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8691a03-666b-4a5b-9adb-eef0d1901c39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:43.742951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-c8691a03-666b-4a5b-9adb-eef0d1901c39 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:43.742951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:43.742951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:43.742951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:43.742951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:43.742951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:43.742951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:43.742951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:43.742951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:43.742951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:43.742951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:43.742951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:43.742951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:43.742951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:43.742951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:43.742951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:43.744195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:43.744195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:43.744195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:43.744195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:43.744195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:43.744195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:43.744195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:43.744195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:43.744195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:43.744195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:43.744195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:43.744195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:43.744195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:43.744195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:43.744195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:43.744195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:43.745246 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:43.745246 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:43.745246 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:43.745246 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:43.745246 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:43.745246 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:43.745246 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:43.745246 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:43.745246 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:43.745246 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:43.745246 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:43.745246 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:43.752570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 345/681] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 30 18:15:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:43.754158 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8691a03-666b-4a5b-9adb-eef0d1901c39 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:43.780596 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9295497-1a8a-4d44-a3f9-d5dd5bf62ad1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:43.824010 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36803c99-f6bc-4cc7-8c81-f4b6243b52dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:43.825128 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36803c99-f6bc-4cc7-8c81-f4b6243b52dc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:43.825656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36803c99-f6bc-4cc7-8c81-f4b6243b52dc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:43.826137 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36803c99-f6bc-4cc7-8c81-f4b6243b52dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:43.844313 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36803c99-f6bc-4cc7-8c81-f4b6243b52dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:43.847590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36803c99-f6bc-4cc7-8c81-f4b6243b52dc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:43.869925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-36803c99-f6bc-4cc7-8c81-f4b6243b52dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:43.869925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:43.869925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:43.869925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:43.869925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:43.869925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:43.869925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:43.869925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:43.869925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:43.869925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:43.869925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:43.869925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:43.869925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:43.869925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:43.869925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:43.870884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:43.870884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:43.870884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:43.870884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:43.870884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:43.870884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:43.870884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:43.870884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:43.870884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:43.870884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:43.870884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:43.870884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:43.870884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:43.870884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:43.870884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:43.870884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:43.870884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:43.871817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:43.871817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:43.871817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:43.871817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:43.871817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 337/682] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:43.872150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36803c99-f6bc-4cc7-8c81-f4b6243b52dc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:43.929413 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8691a03-666b-4a5b-9adb-eef0d1901c39 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:43.974057 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a724c88-c622-4415-9ce0-5387241fe5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:43.975926 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a724c88-c622-4415-9ce0-5387241fe5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:43.976222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a724c88-c622-4415-9ce0-5387241fe5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:43.976504 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a724c88-c622-4415-9ce0-5387241fe5f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:43.977378 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a724c88-c622-4415-9ce0-5387241fe5f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:43.977716 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a724c88-c622-4415-9ce0-5387241fe5f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:43.979369 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a724c88-c622-4415-9ce0-5387241fe5f2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:43.994725 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 346/683] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:15:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:43.995382 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a724c88-c622-4415-9ce0-5387241fe5f2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:44.012058 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36803c99-f6bc-4cc7-8c81-f4b6243b52dc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:44.027741 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:44.051685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66177baa-4fc5-4bb5-bf48-133abc60a156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:44.052665 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66177baa-4fc5-4bb5-bf48-133abc60a156 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:44.053046 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66177baa-4fc5-4bb5-bf48-133abc60a156 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:44.053360 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66177baa-4fc5-4bb5-bf48-133abc60a156 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:44.057603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66177baa-4fc5-4bb5-bf48-133abc60a156 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:44.057603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66177baa-4fc5-4bb5-bf48-133abc60a156 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:44.057603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66177baa-4fc5-4bb5-bf48-133abc60a156 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:44.066254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 338/684] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 30 18:15:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:44.067343 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66177baa-4fc5-4bb5-bf48-133abc60a156 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:44.068207 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:44.119982 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a724c88-c622-4415-9ce0-5387241fe5f2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:44.160630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef7938-d49b-4a2f-abc6-1f56782197a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:44.162226 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef7938-d49b-4a2f-abc6-1f56782197a4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:44.162951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef7938-d49b-4a2f-abc6-1f56782197a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:44.163605 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef7938-d49b-4a2f-abc6-1f56782197a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:44.164907 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef7938-d49b-4a2f-abc6-1f56782197a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:44.165526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef7938-d49b-4a2f-abc6-1f56782197a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:44.167471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef7938-d49b-4a2f-abc6-1f56782197a4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:44.181013 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66177baa-4fc5-4bb5-bf48-133abc60a156 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:44.191357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 347/685] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:44.194285 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef7938-d49b-4a2f-abc6-1f56782197a4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:44.221662 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a7bbd-7ccc-4a33-b666-dfa2a5dbd793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:44.222359 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a7bbd-7ccc-4a33-b666-dfa2a5dbd793 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:44.222618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a7bbd-7ccc-4a33-b666-dfa2a5dbd793 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:44.222951 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a7bbd-7ccc-4a33-b666-dfa2a5dbd793 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:44.223825 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a7bbd-7ccc-4a33-b666-dfa2a5dbd793 None admin] RBAC: Adding query filter params (role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d, scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b, user.id=1d8c54102fb340859ed0acc3189fe376) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:44.224841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a7bbd-7ccc-4a33-b666-dfa2a5dbd793 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:44.226295 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a7bbd-7ccc-4a33-b666-dfa2a5dbd793 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:44.233817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 339/686] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:15:44 2022] GET /identity/v3/role_assignments?user.id=1d8c54102fb340859ed0acc3189fe376&scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b&role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:44.234752 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a7bbd-7ccc-4a33-b666-dfa2a5dbd793 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:44.284981 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:44.381490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a7bbd-7ccc-4a33-b666-dfa2a5dbd793 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:44.442302 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef7938-d49b-4a2f-abc6-1f56782197a4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:44.451537 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46d046-9061-4485-97e4-cb1d45377d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:44.451537 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46d046-9061-4485-97e4-cb1d45377d9f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:44.451537 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46d046-9061-4485-97e4-cb1d45377d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:44.451537 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46d046-9061-4485-97e4-cb1d45377d9f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:44.456632 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f46d046-9061-4485-97e4-cb1d45377d9f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:44.458221 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f46d046-9061-4485-97e4-cb1d45377d9f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:44.478011 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-7f46d046-9061-4485-97e4-cb1d45377d9f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 18:15:44.478011 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:44.478011 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:44.478011 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:44.478011 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:44.478011 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:44.478011 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:44.478011 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:44.478011 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:44.478011 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:44.478011 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:44.478011 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:44.478011 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:15:44.478011 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:15:44.478011 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:15:44.478011 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:15:44.481336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:44.481336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:44.481336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:15:44.481336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:44.481336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:15:44.481336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:44.481336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:44.481336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:44.481336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:44.481336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:44.481336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:44.481336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:44.481336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:44.481336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:44.481336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:44.481336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:44.481336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:44.481336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:44.482543 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:44.482543 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:44.482543 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:15:44.482543 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:15:44.482543 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:15:44.482543 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:15:44.482543 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:15:44.482543 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:15:44.482543 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 18:15:44.482543 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:44.486559 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 340/687] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 30 18:15:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:44.490798 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:44.495584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46d046-9061-4485-97e4-cb1d45377d9f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:44.526473 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f240db0-d6c2-4377-b251-96d0f0b18a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:44.527306 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f240db0-d6c2-4377-b251-96d0f0b18a13 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:44.527596 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f240db0-d6c2-4377-b251-96d0f0b18a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:44.527881 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f240db0-d6c2-4377-b251-96d0f0b18a13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:44.548236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f240db0-d6c2-4377-b251-96d0f0b18a13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:44.550858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f240db0-d6c2-4377-b251-96d0f0b18a13 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:44.577478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-2f240db0-d6c2-4377-b251-96d0f0b18a13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:44.577478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:44.577478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:44.577478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:44.577478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:44.577478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:44.577478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:44.577478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:44.577478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:44.577478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:44.577478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:44.577478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:44.577478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:44.577478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:44.577478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:44.578507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:44.578507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:44.578507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:44.578507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:44.578507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:44.578507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:44.578507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:44.578507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:44.578507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:44.578507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:44.578507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:44.578507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:44.578507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:44.578507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:44.578507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:44.578507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:44.578507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:44.579392 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:44.579392 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:44.579392 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:44.579392 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:44.584357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 348/688] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:44.586219 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f240db0-d6c2-4377-b251-96d0f0b18a13 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:44.657065 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:44.669130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46d046-9061-4485-97e4-cb1d45377d9f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:44.737238 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5cb5bb-f79e-49b1-b039-e592b7516007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:44.737978 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5cb5bb-f79e-49b1-b039-e592b7516007 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:44.738450 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5cb5bb-f79e-49b1-b039-e592b7516007 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:44.738889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5cb5bb-f79e-49b1-b039-e592b7516007 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:44.739756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5cb5bb-f79e-49b1-b039-e592b7516007 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:44.740595 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5cb5bb-f79e-49b1-b039-e592b7516007 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:44.742142 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5cb5bb-f79e-49b1-b039-e592b7516007 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:44.752577 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f240db0-d6c2-4377-b251-96d0f0b18a13 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:44.764805 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 341/689] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 30 18:15:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:44.772515 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5cb5bb-f79e-49b1-b039-e592b7516007 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:44.805052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748f10d-195f-4ac0-a2b1-a2e30d13d4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:44.805758 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748f10d-195f-4ac0-a2b1-a2e30d13d4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:44.806051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748f10d-195f-4ac0-a2b1-a2e30d13d4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:44.807773 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748f10d-195f-4ac0-a2b1-a2e30d13d4d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:44.808555 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5748f10d-195f-4ac0-a2b1-a2e30d13d4d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:44.808877 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5748f10d-195f-4ac0-a2b1-a2e30d13d4d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:44.813216 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5748f10d-195f-4ac0-a2b1-a2e30d13d4d5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:44.842151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 349/690] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:15:44.850440 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748f10d-195f-4ac0-a2b1-a2e30d13d4d5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:44.901792 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:44.937892 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5cb5bb-f79e-49b1-b039-e592b7516007 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:44.995096 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a79781-8043-4733-9e2e-4fa212b5644d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:44.996185 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a79781-8043-4733-9e2e-4fa212b5644d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:44.996675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a79781-8043-4733-9e2e-4fa212b5644d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:44.997159 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a79781-8043-4733-9e2e-4fa212b5644d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:45.037807 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748f10d-195f-4ac0-a2b1-a2e30d13d4d5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:45.061522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:45.099088 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ade99-f3c9-4c60-8b52-d76b80328a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:45.100576 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ade99-f3c9-4c60-8b52-d76b80328a80 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:45.100944 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ade99-f3c9-4c60-8b52-d76b80328a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:45.101323 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ade99-f3c9-4c60-8b52-d76b80328a80 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:45.115904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a79781-8043-4733-9e2e-4fa212b5644d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:45.117451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a79781-8043-4733-9e2e-4fa212b5644d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:45.127262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ade99-f3c9-4c60-8b52-d76b80328a80 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:45.128830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ade99-f3c9-4c60-8b52-d76b80328a80 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:45.159707 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-e19ade99-f3c9-4c60-8b52-d76b80328a80 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 18:15:45.159707 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:45.159707 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:45.159707 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:45.159707 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:45.159707 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:45.159707 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:45.159707 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:45.159707 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:45.159707 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:45.159707 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:45.159707 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:45.159707 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:15:45.159707 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:15:45.159707 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:15:45.159707 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:15:45.160803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:45.160803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:45.160803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:15:45.160803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:45.160803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:15:45.160803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:45.160803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:45.160803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:45.160803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:45.160803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:45.160803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:45.160803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:45.160803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:45.160803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:45.160803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:45.160803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:45.160803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:45.160803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:45.162359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:45.162359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:45.162359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:15:45.162359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:15:45.162359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:15:45.162359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:15:45.162359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:15:45.162359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:15:45.162359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 18:15:45.162359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:45.162359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 350/691] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:15:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:45.167477 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ade99-f3c9-4c60-8b52-d76b80328a80 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:45.244269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 342/692] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:15:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 18:15:45.252977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a79781-8043-4733-9e2e-4fa212b5644d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:45.347062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:45.370103 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ade99-f3c9-4c60-8b52-d76b80328a80 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:45.382126 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:45.408991 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dc3b8c-cf76-4e1d-8a2e-04f2415af30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:45.409714 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc3b8c-cf76-4e1d-8a2e-04f2415af30a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:45.410005 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc3b8c-cf76-4e1d-8a2e-04f2415af30a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:45.410290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc3b8c-cf76-4e1d-8a2e-04f2415af30a None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:45.411155 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dc3b8c-cf76-4e1d-8a2e-04f2415af30a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:45.411476 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dc3b8c-cf76-4e1d-8a2e-04f2415af30a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:45.413025 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dc3b8c-cf76-4e1d-8a2e-04f2415af30a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:45.437775 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 351/693] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Jun 30 18:15:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:45.439360 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dc3b8c-cf76-4e1d-8a2e-04f2415af30a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:45.497687 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a79781-8043-4733-9e2e-4fa212b5644d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:45.554149 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dc3b8c-cf76-4e1d-8a2e-04f2415af30a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:45.586237 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c3bbd-f097-49c5-80b7-b814a5c3dca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:45.587129 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c3bbd-f097-49c5-80b7-b814a5c3dca5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:45.587418 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c3bbd-f097-49c5-80b7-b814a5c3dca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:45.587695 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c3bbd-f097-49c5-80b7-b814a5c3dca5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:45.591639 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00092e77-21b6-41e5-bdf7-4b5cf498a5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:45.592385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00092e77-21b6-41e5-bdf7-4b5cf498a5fd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:45.592666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00092e77-21b6-41e5-bdf7-4b5cf498a5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:45.592977 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00092e77-21b6-41e5-bdf7-4b5cf498a5fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:45.611001 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:45.611954 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c3bbd-f097-49c5-80b7-b814a5c3dca5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:45.617820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c3bbd-f097-49c5-80b7-b814a5c3dca5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:45.656334 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f53c3bbd-f097-49c5-80b7-b814a5c3dca5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:45.656334 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:45.656334 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:45.656334 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:45.656334 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:45.656334 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:45.656334 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:45.656334 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:45.656334 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:45.656334 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:45.656334 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:45.656334 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:45.656334 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:45.656334 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:45.656334 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:45.657336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:45.657336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:45.657336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:45.657336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:45.657336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:45.657336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:45.657336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:45.657336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:45.657336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:45.657336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:45.657336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:45.657336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:45.657336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:45.657336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:45.657336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:45.657336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:45.657336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:45.658349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:45.658349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:45.658349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:45.658349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:45.658349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 343/694] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:45.667285 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c3bbd-f097-49c5-80b7-b814a5c3dca5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:45.672501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00092e77-21b6-41e5-bdf7-4b5cf498a5fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:45.674078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00092e77-21b6-41e5-bdf7-4b5cf498a5fd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:45.791014 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 352/695] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:15:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 18:15:45.792485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d3a5b-c696-4eba-bc56-8e4738089d8a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:45.793204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d3a5b-c696-4eba-bc56-8e4738089d8a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:45.793503 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d3a5b-c696-4eba-bc56-8e4738089d8a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:45.796774 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 353/696] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:45.798131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d3a5b-c696-4eba-bc56-8e4738089d8a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:45.863340 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:45.871909 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c3bbd-f097-49c5-80b7-b814a5c3dca5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:45.884511 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:45.915439 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda1b4c-7b6a-43b1-8186-348b6af43540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:45.916113 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda1b4c-7b6a-43b1-8186-348b6af43540 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:45.916399 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda1b4c-7b6a-43b1-8186-348b6af43540 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:45.916953 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda1b4c-7b6a-43b1-8186-348b6af43540 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:45.917498 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda1b4c-7b6a-43b1-8186-348b6af43540 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:45.917860 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda1b4c-7b6a-43b1-8186-348b6af43540 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:45.919301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda1b4c-7b6a-43b1-8186-348b6af43540 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:45.933453 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 344/697] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:45.934968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f79d9-e3c2-4025-a638-11246432b139 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:45.935262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f79d9-e3c2-4025-a638-11246432b139 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:45.935599 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f79d9-e3c2-4025-a638-11246432b139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:45.938468 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d3a5b-c696-4eba-bc56-8e4738089d8a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:45.955579 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-113f79d9-e3c2-4025-a638-11246432b139 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:45.956939 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-113f79d9-e3c2-4025-a638-11246432b139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:45.985178 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02038798-ac93-42a8-b89b-ab5214c5bcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:45.986859 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02038798-ac93-42a8-b89b-ab5214c5bcac None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:45.987330 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02038798-ac93-42a8-b89b-ab5214c5bcac None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:45.987915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02038798-ac93-42a8-b89b-ab5214c5bcac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:46.001817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02038798-ac93-42a8-b89b-ab5214c5bcac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:46.003563 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02038798-ac93-42a8-b89b-ab5214c5bcac None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:46.028850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-02038798-ac93-42a8-b89b-ab5214c5bcac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:46.028850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:46.028850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:46.028850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:46.028850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:46.028850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:46.028850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:46.028850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:46.028850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:46.028850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:46.028850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:46.028850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:46.028850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:46.028850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:46.028850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:46.029889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:46.029889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:46.029889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:46.029889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:46.029889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:46.029889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:46.029889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:46.029889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:46.029889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:46.029889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:46.029889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:46.029889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:46.029889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:46.029889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:46.029889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:46.029889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:46.029889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:46.030860 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:46.030860 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:46.030860 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:46.030860 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:46.039607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 354/698] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:15:46.039607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02038798-ac93-42a8-b89b-ab5214c5bcac None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:46.047992 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 345/699] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:46.048805 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f79d9-e3c2-4025-a638-11246432b139 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:46.178673 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:46.180949 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:46.182759 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02038798-ac93-42a8-b89b-ab5214c5bcac None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:46.188008 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f79d9-e3c2-4025-a638-11246432b139 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:46.223659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f860743-d1a3-4092-8a11-58a4137d03aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:46.224719 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f860743-d1a3-4092-8a11-58a4137d03aa None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:46.225080 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f860743-d1a3-4092-8a11-58a4137d03aa None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:46.225446 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f860743-d1a3-4092-8a11-58a4137d03aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:46.233326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f860743-d1a3-4092-8a11-58a4137d03aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:46.235160 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f860743-d1a3-4092-8a11-58a4137d03aa None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:46.235561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc56b6-41fb-4a2c-9b6b-560feacc8753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:46.236444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc56b6-41fb-4a2c-9b6b-560feacc8753 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:46.236925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc56b6-41fb-4a2c-9b6b-560feacc8753 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:46.237727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc56b6-41fb-4a2c-9b6b-560feacc8753 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:46.239175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc56b6-41fb-4a2c-9b6b-560feacc8753 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:46.239803 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc56b6-41fb-4a2c-9b6b-560feacc8753 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:46.242816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc56b6-41fb-4a2c-9b6b-560feacc8753 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:46.252242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-6f860743-d1a3-4092-8a11-58a4137d03aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:46.252242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:46.252242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:46.252242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:46.252242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:46.252242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:46.252242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:46.252242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:46.252242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:46.252242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:46.252242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:46.252242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:46.252242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:15:46.252242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:15:46.252242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:15:46.253218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:15:46.253218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:46.253218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:46.253218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:46.253218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:46.253218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:46.253218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:46.253218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:46.253218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:46.253218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:46.253218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:46.253218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:46.253218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:46.253218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:46.253218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:46.253218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:46.253218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:46.254321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:15:46.254321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:15:46.254321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:15:46.254321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:15:46.254321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:15:46.254321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:15:46.254321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:46.254321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:46.254321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 355/700] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:46.255864 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f860743-d1a3-4092-8a11-58a4137d03aa None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:46.266885 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 346/701] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:46.269104 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc56b6-41fb-4a2c-9b6b-560feacc8753 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:46.395483 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f860743-d1a3-4092-8a11-58a4137d03aa None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:46.420012 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc56b6-41fb-4a2c-9b6b-560feacc8753 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:46.444681 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198c7e4-d991-4cef-bed4-2108cb5e73fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:46.446098 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198c7e4-d991-4cef-bed4-2108cb5e73fb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:46.446538 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198c7e4-d991-4cef-bed4-2108cb5e73fb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:46.447214 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198c7e4-d991-4cef-bed4-2108cb5e73fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:46.459534 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6633b3-d465-434d-b6a9-a9cdcfb41fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:46.460373 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6633b3-d465-434d-b6a9-a9cdcfb41fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:46.460760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6633b3-d465-434d-b6a9-a9cdcfb41fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:46.461142 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6633b3-d465-434d-b6a9-a9cdcfb41fc4 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:46.462140 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6633b3-d465-434d-b6a9-a9cdcfb41fc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:46.462560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6633b3-d465-434d-b6a9-a9cdcfb41fc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:46.464139 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198c7e4-d991-4cef-bed4-2108cb5e73fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:46.464286 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6633b3-d465-434d-b6a9-a9cdcfb41fc4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:46.465904 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198c7e4-d991-4cef-bed4-2108cb5e73fb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:46.479186 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 347/702] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:15:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:46.480229 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6633b3-d465-434d-b6a9-a9cdcfb41fc4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:46.491471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-6198c7e4-d991-4cef-bed4-2108cb5e73fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:46.491471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:46.491471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:46.491471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:46.491471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:46.491471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:46.491471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:46.491471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:46.491471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:46.491471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:46.491471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:46.491471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:46.491471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:46.491471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:46.491471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:46.492710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:46.492710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:46.492710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:46.492710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:46.492710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:46.492710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:46.492710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:46.492710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:46.492710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:46.492710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:46.492710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:46.492710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:46.492710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:46.492710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:46.492710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:46.492710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:46.492710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:46.493870 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:46.493870 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:46.493870 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:46.493870 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:46.496795 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 356/703] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:46.497902 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198c7e4-d991-4cef-bed4-2108cb5e73fb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:46.510857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:46.512923 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:46.634401 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6633b3-d465-434d-b6a9-a9cdcfb41fc4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:46.643698 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198c7e4-d991-4cef-bed4-2108cb5e73fb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:46.680739 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7591cb2-941a-4b8b-9638-fd7e77e890e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:46.681757 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7591cb2-941a-4b8b-9638-fd7e77e890e9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:46.681879 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7591cb2-941a-4b8b-9638-fd7e77e890e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:46.682080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7591cb2-941a-4b8b-9638-fd7e77e890e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:46.683513 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e7a24-e727-423f-b604-4850c69f60b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:46.684176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e7a24-e727-423f-b604-4850c69f60b5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:46.684468 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e7a24-e727-423f-b604-4850c69f60b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:46.684752 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e7a24-e727-423f-b604-4850c69f60b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:46.685616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e7a24-e727-423f-b604-4850c69f60b5 None admin] RBAC: Adding query filter params (role.id=0ef3687af20a443281e3cfdceca3597d, scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b, user.id=b8751a4e92174d18b7b138392d2579a1) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:46.685938 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e7a24-e727-423f-b604-4850c69f60b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:46.687407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e7a24-e727-423f-b604-4850c69f60b5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:46.690367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7591cb2-941a-4b8b-9638-fd7e77e890e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:46.692467 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7591cb2-941a-4b8b-9638-fd7e77e890e9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:46.694001 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 357/704] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:15:46 2022] GET /identity/v3/role_assignments?user.id=b8751a4e92174d18b7b138392d2579a1&scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b&role.id=0ef3687af20a443281e3cfdceca3597d => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:46.698644 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e7a24-e727-423f-b604-4850c69f60b5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:46.708319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b7591cb2-941a-4b8b-9638-fd7e77e890e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:46.708319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:46.708319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:46.708319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:46.708319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:46.708319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:46.708319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:46.708319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:46.708319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:46.708319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:46.708319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:46.708319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:46.708319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:15:46.708319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:15:46.708319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:15:46.709367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:15:46.709367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:46.709367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:46.709367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:46.709367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:46.709367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:46.709367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:46.709367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:46.709367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:46.709367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:46.709367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:46.709367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:46.709367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:46.709367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:46.709367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:46.709367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:46.709367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:46.711041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:15:46.711041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:15:46.711041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:15:46.711041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:15:46.711041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:15:46.711041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:15:46.711041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:46.711041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:46.715370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 348/705] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:46.717305 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ef232-fab6-4ed0-bca8-27c8d26fb636 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:46.719070 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ef232-fab6-4ed0-bca8-27c8d26fb636 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:46.719070 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ef232-fab6-4ed0-bca8-27c8d26fb636 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:46.719754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 349/706] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:46.721541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ef232-fab6-4ed0-bca8-27c8d26fb636 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:46.843374 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:46.866787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:46.899786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ef232-fab6-4ed0-bca8-27c8d26fb636 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:46.902423 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e7a24-e727-423f-b604-4850c69f60b5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:46.950015 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be8fc7-c05e-41e4-8f8b-7c3310d0608a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:46.951063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be8fc7-c05e-41e4-8f8b-7c3310d0608a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:46.951568 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be8fc7-c05e-41e4-8f8b-7c3310d0608a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:46.952120 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be8fc7-c05e-41e4-8f8b-7c3310d0608a None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:46.953150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be8fc7-c05e-41e4-8f8b-7c3310d0608a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:46.953707 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be8fc7-c05e-41e4-8f8b-7c3310d0608a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:46.955539 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be8fc7-c05e-41e4-8f8b-7c3310d0608a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:46.977659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381b2ee-6dc5-4be9-a783-9601fb1abab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:46.977659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381b2ee-6dc5-4be9-a783-9601fb1abab9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:46.977659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381b2ee-6dc5-4be9-a783-9601fb1abab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:46.977659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381b2ee-6dc5-4be9-a783-9601fb1abab9 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:46.978091 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381b2ee-6dc5-4be9-a783-9601fb1abab9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:46.978403 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381b2ee-6dc5-4be9-a783-9601fb1abab9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:46.978701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 350/707] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:15:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:46.980593 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b684d42-e5a0-4a03-804a-ed82a2bc5f8b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:46.980912 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381b2ee-6dc5-4be9-a783-9601fb1abab9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:46.981422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b684d42-e5a0-4a03-804a-ed82a2bc5f8b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:46.981927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b684d42-e5a0-4a03-804a-ed82a2bc5f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:47.003176 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2b684d42-e5a0-4a03-804a-ed82a2bc5f8b None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:47.004811 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2b684d42-e5a0-4a03-804a-ed82a2bc5f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:47.017505 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 358/708] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:47.019193 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381b2ee-6dc5-4be9-a783-9601fb1abab9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:47.134582 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 351/709] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:47.134582 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae67676-c3bf-475c-aaf6-99c6ff14fea4 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:47.134582 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae67676-c3bf-475c-aaf6-99c6ff14fea4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:47.134582 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae67676-c3bf-475c-aaf6-99c6ff14fea4 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:47.134582 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 352/710] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:47.134582 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae67676-c3bf-475c-aaf6-99c6ff14fea4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:47.206664 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381b2ee-6dc5-4be9-a783-9601fb1abab9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:47.246169 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:47.247312 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb28016-b3d7-4ca3-9f76-d494f9e9a102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:47.247914 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb28016-b3d7-4ca3-9f76-d494f9e9a102 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:47.248201 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb28016-b3d7-4ca3-9f76-d494f9e9a102 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:47.248576 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb28016-b3d7-4ca3-9f76-d494f9e9a102 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:47.249379 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb28016-b3d7-4ca3-9f76-d494f9e9a102 None admin] RBAC: Adding query filter params (role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d, scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b, user.id=4bd521b9e93f4b3b8af64d46f36c019e) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:47.249795 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb28016-b3d7-4ca3-9f76-d494f9e9a102 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:47.251305 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb28016-b3d7-4ca3-9f76-d494f9e9a102 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:47.253518 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:47.260144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 359/711] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:15:47 2022] GET /identity/v3/role_assignments?user.id=4bd521b9e93f4b3b8af64d46f36c019e&scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b&role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:47.261520 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15087e-ad43-4d59-b157-b48d8772227e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:47.261842 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15087e-ad43-4d59-b157-b48d8772227e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:47.262122 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15087e-ad43-4d59-b157-b48d8772227e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:47.281564 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4f15087e-ad43-4d59-b157-b48d8772227e None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:47.282866 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4f15087e-ad43-4d59-b157-b48d8772227e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:47.326480 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae67676-c3bf-475c-aaf6-99c6ff14fea4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:47.373757 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75116ee7-aa7c-43f8-98fc-c9a46aa6dbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:47.374465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75116ee7-aa7c-43f8-98fc-c9a46aa6dbbd None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:47.374750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75116ee7-aa7c-43f8-98fc-c9a46aa6dbbd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:47.375053 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75116ee7-aa7c-43f8-98fc-c9a46aa6dbbd None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:47.375767 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75116ee7-aa7c-43f8-98fc-c9a46aa6dbbd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:47.377496 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75116ee7-aa7c-43f8-98fc-c9a46aa6dbbd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:47.398176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 360/712] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:47.398176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15087e-ad43-4d59-b157-b48d8772227e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:47.417948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 353/713] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 30 18:15:47 2022] POST /identity/v3/users => generated 254 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:15:47.420159 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75116ee7-aa7c-43f8-98fc-c9a46aa6dbbd None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:47.561315 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15087e-ad43-4d59-b157-b48d8772227e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:47.576793 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75116ee7-aa7c-43f8-98fc-c9a46aa6dbbd None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:47.601012 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:47.616526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:47.619720 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f53088-52e3-4698-8ae4-31181ce716bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:47.620836 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f53088-52e3-4698-8ae4-31181ce716bd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:47.621195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f53088-52e3-4698-8ae4-31181ce716bd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:47.621545 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f53088-52e3-4698-8ae4-31181ce716bd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:47.629610 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f53088-52e3-4698-8ae4-31181ce716bd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:47.629610 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f53088-52e3-4698-8ae4-31181ce716bd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:47.641957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3d9de-1c0f-4af5-8567-080bc625cb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:47.642779 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3d9de-1c0f-4af5-8567-080bc625cb3e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:47.643057 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3d9de-1c0f-4af5-8567-080bc625cb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:47.643327 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3d9de-1c0f-4af5-8567-080bc625cb3e None admin] PATH_INFO: `/v3/services/network` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:47.644002 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee3d9de-1c0f-4af5-8567-080bc625cb3e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:47.645424 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee3d9de-1c0f-4af5-8567-080bc625cb3e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:47.649112 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-e0f53088-52e3-4698-8ae4-31181ce716bd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 18:15:47.649112 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:47.649112 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:15:47.649112 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:15:47.649112 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:15:47.649112 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:15:47.649112 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:15:47.649112 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:15:47.649112 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 18:15:47.649112 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:47.649112 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:15:47.649112 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:47.649112 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:47.649112 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:47.649112 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:47.649112 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:47.650211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:47.650211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:47.650211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:47.650211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:47.650211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:47.650211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:47.650211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:47.650211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:15:47.650211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:15:47.650211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:15:47.650211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:15:47.650211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:47.650211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:47.650211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:47.650211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:47.650211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:47.651076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:47.651076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:47.651076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:47.651076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:47.651076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:47.651076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:47.651076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:47.651076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:47.651076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:47.651076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:47.651076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:47.651076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:15:47.651076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:15:47.651076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 18:15:47.651076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:47.651076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 361/714] 10.222.0.33 () {56 vars in 1129 bytes} [Thu Jun 30 18:15:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:47.659815 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f53088-52e3-4698-8ae4-31181ce716bd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:47.667827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-0ee3d9de-1c0f-4af5-8567-080bc625cb3e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 18:15:47.667827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:47.667827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:15:47.667827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:15:47.667827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:15:47.667827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:15:47.667827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:15:47.667827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:15:47.667827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 18:15:47.667827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:47.667827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:15:47.667827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:47.667827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:47.667827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:47.667827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:47.667827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:47.667827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:47.668908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:47.668908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:47.668908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:47.668908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:47.668908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:47.668908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:47.668908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:15:47.668908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:15:47.668908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:15:47.668908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:15:47.668908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:47.668908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:47.668908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:47.668908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:47.668908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:47.668908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:47.670095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:47.670095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:47.670095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:47.670095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:47.670095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:47.670095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:47.670095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:47.670095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:47.670095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:47.670095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:47.670095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:15:47.670095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:15:47.670095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 18:15:47.670095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:47.670095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 354/715] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:47 2022] GET /identity/v3/services/network => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:47.676199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3d9de-1c0f-4af5-8567-080bc625cb3e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:47.824178 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f53088-52e3-4698-8ae4-31181ce716bd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:47.831743 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3d9de-1c0f-4af5-8567-080bc625cb3e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:47.861118 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba65b5-aceb-4a4c-8225-9e501e1eb215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:47.862048 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba65b5-aceb-4a4c-8225-9e501e1eb215 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:47.862048 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba65b5-aceb-4a4c-8225-9e501e1eb215 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:47.862335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba65b5-aceb-4a4c-8225-9e501e1eb215 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:47.863051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba65b5-aceb-4a4c-8225-9e501e1eb215 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:47.863359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba65b5-aceb-4a4c-8225-9e501e1eb215 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:47.864727 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba65b5-aceb-4a4c-8225-9e501e1eb215 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:47.869774 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 362/716] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Jun 30 18:15:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:15:47.872357 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bca720-512a-409a-baad-ac6d9d5e880e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:47.873249 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bca720-512a-409a-baad-ac6d9d5e880e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:47.873774 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bca720-512a-409a-baad-ac6d9d5e880e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:47.874294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bca720-512a-409a-baad-ac6d9d5e880e None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:47.875202 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bca720-512a-409a-baad-ac6d9d5e880e None admin] RBAC: Adding query filter params (name=network) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:47.875710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bca720-512a-409a-baad-ac6d9d5e880e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:47.877292 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bca720-512a-409a-baad-ac6d9d5e880e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:47.883104 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba65b5-aceb-4a4c-8225-9e501e1eb215 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:47.883418 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 355/717] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 30 18:15:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:47.888188 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bca720-512a-409a-baad-ac6d9d5e880e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:47.997995 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:48.020783 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bca720-512a-409a-baad-ac6d9d5e880e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:48.035913 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:48.088079 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206867f-260f-4eac-ad7e-fed9428ad5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:48.090439 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206867f-260f-4eac-ad7e-fed9428ad5fb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:48.090934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206867f-260f-4eac-ad7e-fed9428ad5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:48.091412 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206867f-260f-4eac-ad7e-fed9428ad5fb None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:48.092521 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206867f-260f-4eac-ad7e-fed9428ad5fb None admin] RBAC: Adding query filter params (type=network) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:48.093947 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206867f-260f-4eac-ad7e-fed9428ad5fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:48.096666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206867f-260f-4eac-ad7e-fed9428ad5fb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:48.111916 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 356/718] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 30 18:15:47 2022] GET /identity/v3/services?type=network => generated 124 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:48.127806 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba65b5-aceb-4a4c-8225-9e501e1eb215 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:48.200172 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91c3d5-b8f0-4908-b08b-8ec24750ee69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:48.200863 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91c3d5-b8f0-4908-b08b-8ec24750ee69 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:48.201160 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91c3d5-b8f0-4908-b08b-8ec24750ee69 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:48.201464 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91c3d5-b8f0-4908-b08b-8ec24750ee69 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:48.202217 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb91c3d5-b8f0-4908-b08b-8ec24750ee69 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:48.202534 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb91c3d5-b8f0-4908-b08b-8ec24750ee69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:48.203974 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb91c3d5-b8f0-4908-b08b-8ec24750ee69 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:48.215164 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 363/719] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Jun 30 18:15:47 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:49.282403 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b455a-5ee5-4a97-8391-492791e43a17 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:49.283413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b455a-5ee5-4a97-8391-492791e43a17 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:49.283910 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b455a-5ee5-4a97-8391-492791e43a17 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:49.285782 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 357/720] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:49.292839 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3581e7d-8e14-4925-802a-fb94ebb82725 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:49.293433 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3581e7d-8e14-4925-802a-fb94ebb82725 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:49.293896 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3581e7d-8e14-4925-802a-fb94ebb82725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:49.337315 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d3581e7d-8e14-4925-802a-fb94ebb82725 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:49.339396 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d3581e7d-8e14-4925-802a-fb94ebb82725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:49.499300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 364/721] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:49.536231 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b455a-5ee5-4a97-8391-492791e43a17 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:49.765685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b455a-5ee5-4a97-8391-492791e43a17 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:49.796286 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:49.831611 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0bcb6f-9fd1-4c2f-ac3b-d19c9ddf163c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:49.832952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bcb6f-9fd1-4c2f-ac3b-d19c9ddf163c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:49.833452 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bcb6f-9fd1-4c2f-ac3b-d19c9ddf163c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:49.834509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bcb6f-9fd1-4c2f-ac3b-d19c9ddf163c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:49.860156 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0bcb6f-9fd1-4c2f-ac3b-d19c9ddf163c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:49.875937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4d0bcb6f-9fd1-4c2f-ac3b-d19c9ddf163c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:15:49.875937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:49.875937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:49.875937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:49.875937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:49.875937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:49.875937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:49.875937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:49.875937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:49.875937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:49.875937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:49.875937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:49.875937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:49.875937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:49.875937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:49.875937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:49.876945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:49.876945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:49.876945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:49.876945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:49.876945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:49.876945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:49.876945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:49.876945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:49.876945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:49.876945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:49.876945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:49.876945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:49.876945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:49.876945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:49.876945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:49.876945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:49.877820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:49.877820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:49.877820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:49.877820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:49.877820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:49.877820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:49.877820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:49.877820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:49.877820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:49.877820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:49.877820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:15:49.877820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:49.878949 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 358/722] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:15:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:49.887622 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3581e7d-8e14-4925-802a-fb94ebb82725 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:50.005288 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:50.116738 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3581e7d-8e14-4925-802a-fb94ebb82725 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:50.178705 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015009aa-4b02-4b5e-8ca9-268010aef41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:50.179796 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015009aa-4b02-4b5e-8ca9-268010aef41c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:50.180325 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015009aa-4b02-4b5e-8ca9-268010aef41c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:50.180833 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015009aa-4b02-4b5e-8ca9-268010aef41c None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:50.181855 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015009aa-4b02-4b5e-8ca9-268010aef41c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:50.182401 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015009aa-4b02-4b5e-8ca9-268010aef41c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:50.184065 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015009aa-4b02-4b5e-8ca9-268010aef41c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:50.195190 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 365/723] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:15:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:50.201847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0bcb6f-9fd1-4c2f-ac3b-d19c9ddf163c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:50.447980 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352de0a-a2c7-4f50-8472-b64da1e595a7 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:50.448961 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352de0a-a2c7-4f50-8472-b64da1e595a7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:50.449462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352de0a-a2c7-4f50-8472-b64da1e595a7 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:50.451421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 366/724] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:15:50.457338 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017879a-3ad9-482b-9991-7dd8467be43f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:50.458067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017879a-3ad9-482b-9991-7dd8467be43f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:50.458571 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017879a-3ad9-482b-9991-7dd8467be43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:50.469254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0bcb6f-9fd1-4c2f-ac3b-d19c9ddf163c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:50.484109 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7017879a-3ad9-482b-9991-7dd8467be43f None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:50.486027 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7017879a-3ad9-482b-9991-7dd8467be43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:50.552130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5fdf5-e2be-4d0d-803d-63107c954253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:50.553860 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5fdf5-e2be-4d0d-803d-63107c954253 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:50.554422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5fdf5-e2be-4d0d-803d-63107c954253 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:50.554963 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5fdf5-e2be-4d0d-803d-63107c954253 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:50.577626 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5fdf5-e2be-4d0d-803d-63107c954253 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:50.580243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5fdf5-e2be-4d0d-803d-63107c954253 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:50.599083 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 367/725] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:50.615866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a9f5fdf5-e2be-4d0d-803d-63107c954253 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:50.615866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:50.615866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:50.615866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:50.615866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:50.615866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:50.615866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:50.615866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:50.615866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:50.615866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:50.615866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:50.615866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:50.615866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:50.615866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:50.615866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:50.616830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:50.616830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:50.616830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:50.616830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:50.616830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:50.616830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:50.616830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:50.616830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:50.616830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:50.616830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:50.616830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:50.616830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:50.616830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:50.616830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:50.616830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:50.616830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:50.616830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:50.617748 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:50.617748 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:50.617748 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:50.617748 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:50.620108 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 359/726] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:50.625561 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7017879a-3ad9-482b-9991-7dd8467be43f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:50.645876 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5fdf5-e2be-4d0d-803d-63107c954253 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:50.704956 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:50.714890 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:50.796123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7017879a-3ad9-482b-9991-7dd8467be43f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:50.800284 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5fdf5-e2be-4d0d-803d-63107c954253 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:50.833418 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659d61d-0e42-4144-8636-3c5a23f6a8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:50.835297 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659d61d-0e42-4144-8636-3c5a23f6a8aa None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:50.835585 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659d61d-0e42-4144-8636-3c5a23f6a8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:50.835982 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659d61d-0e42-4144-8636-3c5a23f6a8aa None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:50.836930 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659d61d-0e42-4144-8636-3c5a23f6a8aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:50.837343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659d61d-0e42-4144-8636-3c5a23f6a8aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:50.839008 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a8db9-f975-45f1-b918-c9662f999ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:50.839193 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659d61d-0e42-4144-8636-3c5a23f6a8aa None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:50.839801 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a8db9-f975-45f1-b918-c9662f999ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:50.840145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a8db9-f975-45f1-b918-c9662f999ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:50.840525 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a8db9-f975-45f1-b918-c9662f999ee1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:50.852564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a8db9-f975-45f1-b918-c9662f999ee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:50.857242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 368/727] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:15:50.858677 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758fec9-729e-4b61-a928-6bd4cd74c0ae None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:50.858962 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758fec9-729e-4b61-a928-6bd4cd74c0ae None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:50.859249 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758fec9-729e-4b61-a928-6bd4cd74c0ae None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:50.860612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-609a8db9-f975-45f1-b918-c9662f999ee1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:50.860612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:50.860612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:50.860612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:50.860612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:50.860612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:50.860612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:50.860612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:50.860612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:50.860612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:50.860612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:50.860612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:50.860612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:50.860612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:50.860612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:50.860612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:50.862370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:50.862370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:50.862370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:50.862370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:50.862370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:50.862370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:50.862370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:50.862370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:50.862370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:50.862370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:50.862370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:50.862370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:50.862370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:50.862370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:50.862370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:50.862370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:50.864041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:50.864041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:50.864041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:50.864041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:50.864041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:50.864041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:50.864041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:50.864041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:50.864041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:50.864041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:50.864041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:50.864041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:50.865345 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 369/728] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:50.865345 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758fec9-729e-4b61-a928-6bd4cd74c0ae None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:50.866460 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 360/729] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 30 18:15:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:15:50.873607 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8312fb-04d1-4cb8-8c55-705bc24ef963 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:50.873607 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8312fb-04d1-4cb8-8c55-705bc24ef963 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:50.873607 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8312fb-04d1-4cb8-8c55-705bc24ef963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:50.904977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ab8312fb-04d1-4cb8-8c55-705bc24ef963 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:50.908444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ab8312fb-04d1-4cb8-8c55-705bc24ef963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:50.987183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758fec9-729e-4b61-a928-6bd4cd74c0ae None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:50.988760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:51.037660 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54503c8-529c-4a2b-b114-53395319417b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:51.040379 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54503c8-529c-4a2b-b114-53395319417b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:51.041266 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54503c8-529c-4a2b-b114-53395319417b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:51.042311 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54503c8-529c-4a2b-b114-53395319417b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:51.058362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54503c8-529c-4a2b-b114-53395319417b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:51.059621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 361/730] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 18:15:51.060779 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8312fb-04d1-4cb8-8c55-705bc24ef963 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:51.066729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54503c8-529c-4a2b-b114-53395319417b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:51.098272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f54503c8-529c-4a2b-b114-53395319417b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 18:15:51.098272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:51.098272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:51.098272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:51.098272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:51.098272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:51.098272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:51.098272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:51.098272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:51.098272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:51.098272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:51.098272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:51.098272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:15:51.098272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:15:51.098272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:15:51.098272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:15:51.099356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:51.099356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:51.099356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:15:51.099356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:51.099356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:15:51.099356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:51.099356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:51.099356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:51.099356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:51.099356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:51.099356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:51.099356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:51.099356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:51.099356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:51.099356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:51.099356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:51.099356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:51.099356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:51.100549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:51.100549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:51.100549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:15:51.100549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:15:51.100549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:15:51.100549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:15:51.100549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:15:51.100549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:15:51.100549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 18:15:51.100549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:51.101107 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:51.107934 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 370/731] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 30 18:15:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:15:51.109285 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54503c8-529c-4a2b-b114-53395319417b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:51.201663 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8312fb-04d1-4cb8-8c55-705bc24ef963 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:51.232349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54503c8-529c-4a2b-b114-53395319417b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:51.239342 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5685987-acde-4728-b999-d10d3a0e4001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:51.240012 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5685987-acde-4728-b999-d10d3a0e4001 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:51.240346 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5685987-acde-4728-b999-d10d3a0e4001 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:51.240619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5685987-acde-4728-b999-d10d3a0e4001 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:51.241372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5685987-acde-4728-b999-d10d3a0e4001 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:51.241712 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5685987-acde-4728-b999-d10d3a0e4001 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:51.243102 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5685987-acde-4728-b999-d10d3a0e4001 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:51.257121 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 362/732] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 30 18:15:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:51.257121 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5685987-acde-4728-b999-d10d3a0e4001 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:51.279148 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb23ba-f166-497b-bd8f-225f2625a546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:51.279930 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb23ba-f166-497b-bd8f-225f2625a546 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:51.280216 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb23ba-f166-497b-bd8f-225f2625a546 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:51.280542 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb23ba-f166-497b-bd8f-225f2625a546 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:51.291523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb23ba-f166-497b-bd8f-225f2625a546 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:51.299463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-29eb23ba-f166-497b-bd8f-225f2625a546 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:15:51.299463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:51.299463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:51.299463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:51.299463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:51.299463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:51.299463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:51.299463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:51.299463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:51.299463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:51.299463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:51.299463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:51.299463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:51.299463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:51.299463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:51.299463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:51.300442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:51.300442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:51.300442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:51.300442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:51.300442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:51.300442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:51.300442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:51.300442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:51.300442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:51.300442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:51.300442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:51.300442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:51.300442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:51.300442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:51.300442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:51.300442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:51.301290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:51.301290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:51.301290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:51.301290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:51.301290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:51.301290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:51.301290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:51.301290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:51.301290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:51.301290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:51.301290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:15:51.301290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:51.301290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 371/733] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:15:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:51.302050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb23ba-f166-497b-bd8f-225f2625a546 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:51.332747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:51.353208 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:51.375087 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5685987-acde-4728-b999-d10d3a0e4001 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:51.412385 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505f727-b8be-4a98-a1ce-d402a7049fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:51.413073 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505f727-b8be-4a98-a1ce-d402a7049fba None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:51.413361 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505f727-b8be-4a98-a1ce-d402a7049fba None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:51.413737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505f727-b8be-4a98-a1ce-d402a7049fba None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:51.414697 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505f727-b8be-4a98-a1ce-d402a7049fba None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:51.415018 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505f727-b8be-4a98-a1ce-d402a7049fba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:51.416595 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505f727-b8be-4a98-a1ce-d402a7049fba None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:51.422125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb23ba-f166-497b-bd8f-225f2625a546 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:51.439238 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 363/734] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 30 18:15:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:51.440575 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7e901-2e34-40fb-b9c9-fe6fc9a10de4 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:51.440859 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7e901-2e34-40fb-b9c9-fe6fc9a10de4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:51.441134 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7e901-2e34-40fb-b9c9-fe6fc9a10de4 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:51.447196 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 364/735] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:51.447196 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7e901-2e34-40fb-b9c9-fe6fc9a10de4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:51.467516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ad136-5eb9-4509-a0df-50958ca84c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:51.468473 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ad136-5eb9-4509-a0df-50958ca84c19 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:51.468758 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ad136-5eb9-4509-a0df-50958ca84c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:51.469337 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ad136-5eb9-4509-a0df-50958ca84c19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:51.486289 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ad136-5eb9-4509-a0df-50958ca84c19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:51.488768 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ad136-5eb9-4509-a0df-50958ca84c19 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:51.511316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5e6ad136-5eb9-4509-a0df-50958ca84c19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:51.511316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:51.511316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:51.511316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:51.511316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:51.511316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:51.511316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:51.511316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:51.511316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:51.511316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:51.511316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:51.511316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:51.511316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:51.511316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:51.511316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:51.512250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:51.512250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:51.512250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:51.512250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:51.512250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:51.512250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:51.512250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:51.512250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:51.512250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:51.512250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:51.512250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:51.512250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:51.512250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:51.512250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:51.512250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:51.512250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:51.512250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:51.513143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:51.513143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:51.513143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:51.513143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:51.513143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 372/736] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:51.513976 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ad136-5eb9-4509-a0df-50958ca84c19 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:51.566845 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7e901-2e34-40fb-b9c9-fe6fc9a10de4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:51.592893 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:51.603906 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5410b843-acef-4d15-9d92-755428f6a2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:51.604669 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410b843-acef-4d15-9d92-755428f6a2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:51.604926 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410b843-acef-4d15-9d92-755428f6a2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:51.605207 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410b843-acef-4d15-9d92-755428f6a2b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:51.606002 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5410b843-acef-4d15-9d92-755428f6a2b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:51.606310 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5410b843-acef-4d15-9d92-755428f6a2b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:51.607721 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5410b843-acef-4d15-9d92-755428f6a2b9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:51.612999 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:51.617117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 365/737] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:15:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:51.619231 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4537697-0661-4a89-9b00-615a1df3c9fd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:51.619519 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4537697-0661-4a89-9b00-615a1df3c9fd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:51.619794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4537697-0661-4a89-9b00-615a1df3c9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:51.621889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ad136-5eb9-4509-a0df-50958ca84c19 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:51.637121 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e4537697-0661-4a89-9b00-615a1df3c9fd None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:51.638360 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e4537697-0661-4a89-9b00-615a1df3c9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:51.662779 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e2c2a-fdc5-494d-aeba-1ece421df3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:51.663663 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e2c2a-fdc5-494d-aeba-1ece421df3c5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:51.663867 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e2c2a-fdc5-494d-aeba-1ece421df3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:51.664151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e2c2a-fdc5-494d-aeba-1ece421df3c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:51.678843 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e2c2a-fdc5-494d-aeba-1ece421df3c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:51.680342 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e2c2a-fdc5-494d-aeba-1ece421df3c5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:51.699304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-a50e2c2a-fdc5-494d-aeba-1ece421df3c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:51.699304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:51.699304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:51.699304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:51.699304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:51.699304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:51.699304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:51.699304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:51.699304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:51.699304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:51.699304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:51.699304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:51.699304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:51.699304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:51.699304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:51.700384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:51.700384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:51.700384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:51.700384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:51.700384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:51.700384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:51.700384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:51.700384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:51.700384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:51.700384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:51.700384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:51.700384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:51.700384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:51.700384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:51.700384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:51.700384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:51.700384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:51.701309 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:51.701309 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:51.701309 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:51.701309 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:51.701309 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 373/738] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:51.701628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e2c2a-fdc5-494d-aeba-1ece421df3c5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:51.721279 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 366/739] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:51.722772 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7401c-994d-488e-96a3-9afe0d68c1de None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:51.723068 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7401c-994d-488e-96a3-9afe0d68c1de None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:51.723346 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7401c-994d-488e-96a3-9afe0d68c1de None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:51.724673 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 367/740] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:51.725380 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7401c-994d-488e-96a3-9afe0d68c1de None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:51.825971 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e2c2a-fdc5-494d-aeba-1ece421df3c5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:51.855919 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:51.860620 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7401c-994d-488e-96a3-9afe0d68c1de None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:51.869211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd415d-7ab1-43e9-98fc-8a488d90060a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:51.870421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd415d-7ab1-43e9-98fc-8a488d90060a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:51.870925 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd415d-7ab1-43e9-98fc-8a488d90060a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:51.871444 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd415d-7ab1-43e9-98fc-8a488d90060a None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:51.872828 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd415d-7ab1-43e9-98fc-8a488d90060a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:51.873401 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd415d-7ab1-43e9-98fc-8a488d90060a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:51.876126 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd415d-7ab1-43e9-98fc-8a488d90060a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:51.889339 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 374/741] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:51.890302 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd415d-7ab1-43e9-98fc-8a488d90060a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:51.892760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:51.900219 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4136e0-468a-4179-8393-16de239abf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:51.901005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4136e0-468a-4179-8393-16de239abf30 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:51.901284 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4136e0-468a-4179-8393-16de239abf30 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:51.901554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4136e0-468a-4179-8393-16de239abf30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:51.911565 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4136e0-468a-4179-8393-16de239abf30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:51.913165 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4136e0-468a-4179-8393-16de239abf30 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:51.931564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-1e4136e0-468a-4179-8393-16de239abf30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:51.931564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:51.931564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:51.931564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:51.931564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:51.931564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:51.931564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:51.931564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:51.931564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:51.931564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:51.931564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:51.931564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:51.931564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:51.931564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:51.931564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:51.932675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:51.932675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:51.932675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:51.932675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:51.932675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:51.932675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:51.932675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:51.932675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:51.932675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:51.932675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:51.932675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:51.932675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:51.932675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:51.932675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:51.932675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:51.932675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:51.932675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:51.933635 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:51.933635 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:51.933635 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:51.933635 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:51.933635 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 368/742] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:51.934084 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4983b3-f77b-4444-ac96-3f45cc85b934 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:51.934367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4983b3-f77b-4444-ac96-3f45cc85b934 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:51.934642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4983b3-f77b-4444-ac96-3f45cc85b934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:51.952476 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-bb4983b3-f77b-4444-ac96-3f45cc85b934 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:51.953711 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bb4983b3-f77b-4444-ac96-3f45cc85b934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:51.999619 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd415d-7ab1-43e9-98fc-8a488d90060a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:52.027553 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 369/743] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 18:15:52.035026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4983b3-f77b-4444-ac96-3f45cc85b934 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:52.048717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e9cbf-9c46-4a00-8352-acbf343ced61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:52.050363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e9cbf-9c46-4a00-8352-acbf343ced61 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:52.050652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e9cbf-9c46-4a00-8352-acbf343ced61 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:52.050940 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e9cbf-9c46-4a00-8352-acbf343ced61 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:52.051773 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e9cbf-9c46-4a00-8352-acbf343ced61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:52.052095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e9cbf-9c46-4a00-8352-acbf343ced61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:52.054710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e9cbf-9c46-4a00-8352-acbf343ced61 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:52.067294 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 375/744] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:52.068663 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e9cbf-9c46-4a00-8352-acbf343ced61 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:52.114681 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:52.135655 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4983b3-f77b-4444-ac96-3f45cc85b934 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:52.149176 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:52.170327 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e9cbf-9c46-4a00-8352-acbf343ced61 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:52.172567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f945d7b-01d5-4716-8c00-9517205d80fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:52.173288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f945d7b-01d5-4716-8c00-9517205d80fe None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:52.173565 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f945d7b-01d5-4716-8c00-9517205d80fe None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:52.173877 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f945d7b-01d5-4716-8c00-9517205d80fe None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:52.174611 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f945d7b-01d5-4716-8c00-9517205d80fe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:52.176031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f945d7b-01d5-4716-8c00-9517205d80fe None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:52.192847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 370/745] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 30 18:15:52 2022] POST /identity/v3/services => generated 251 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:15:52.195138 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f945d7b-01d5-4716-8c00-9517205d80fe None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:52.229407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b47373b-017d-4a6a-a9ec-c8bfeb58baef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:52.231218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47373b-017d-4a6a-a9ec-c8bfeb58baef None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:52.231729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47373b-017d-4a6a-a9ec-c8bfeb58baef None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:52.232246 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47373b-017d-4a6a-a9ec-c8bfeb58baef None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:52.245286 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b47373b-017d-4a6a-a9ec-c8bfeb58baef None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:52.247565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b47373b-017d-4a6a-a9ec-c8bfeb58baef None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:52.270528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5b47373b-017d-4a6a-a9ec-c8bfeb58baef None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 18:15:52.270528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:52.270528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:52.270528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:52.270528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:52.270528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:52.270528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:52.270528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:52.270528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:52.270528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:52.270528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:52.270528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:52.270528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:15:52.270528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:15:52.270528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:15:52.270528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:15:52.271790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:52.271790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:52.271790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:15:52.271790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:52.271790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:15:52.271790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:52.271790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:52.271790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:52.271790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:52.271790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:52.271790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:52.271790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:52.271790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:52.271790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:52.271790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:52.271790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:52.271790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:52.271790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:52.272729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:52.272729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:52.272729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:15:52.272729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:15:52.272729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:15:52.272729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:15:52.272729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:15:52.272729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:15:52.272729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 18:15:52.272729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:52.272729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 376/746] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 30 18:15:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:52.274102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b47373b-017d-4a6a-a9ec-c8bfeb58baef None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:52.370790 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f945d7b-01d5-4716-8c00-9517205d80fe None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:52.429696 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58b28d-a811-4743-953f-b3f0250e049f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:52.430823 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58b28d-a811-4743-953f-b3f0250e049f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:52.431287 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58b28d-a811-4743-953f-b3f0250e049f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:52.431784 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58b28d-a811-4743-953f-b3f0250e049f None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:52.433196 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d58b28d-a811-4743-953f-b3f0250e049f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:52.433779 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d58b28d-a811-4743-953f-b3f0250e049f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:52.436514 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b47373b-017d-4a6a-a9ec-c8bfeb58baef None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:52.436713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d58b28d-a811-4743-953f-b3f0250e049f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:52.438589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:52.453629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 371/747] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:52.453832 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58b28d-a811-4743-953f-b3f0250e049f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:52.495478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad33db-f22a-4b0f-bbdc-90577c79b5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:52.497563 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad33db-f22a-4b0f-bbdc-90577c79b5ae None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:52.498088 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad33db-f22a-4b0f-bbdc-90577c79b5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:52.498571 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad33db-f22a-4b0f-bbdc-90577c79b5ae None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:52.500172 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad33db-f22a-4b0f-bbdc-90577c79b5ae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:52.502308 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad33db-f22a-4b0f-bbdc-90577c79b5ae None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:52.504094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:52.526812 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 377/748] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 30 18:15:52 2022] POST /identity/v3/services => generated 240 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:15:52.527664 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad33db-f22a-4b0f-bbdc-90577c79b5ae None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:52.663237 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58b28d-a811-4743-953f-b3f0250e049f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:52.692048 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad33db-f22a-4b0f-bbdc-90577c79b5ae None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:52.717623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf2608f-920c-4055-9d37-766e3e5e2972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:52.719769 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf2608f-920c-4055-9d37-766e3e5e2972 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:52.719769 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf2608f-920c-4055-9d37-766e3e5e2972 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:52.720108 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf2608f-920c-4055-9d37-766e3e5e2972 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:52.732764 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf2608f-920c-4055-9d37-766e3e5e2972 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:52.735191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf2608f-920c-4055-9d37-766e3e5e2972 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:52.755202 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0cbd6a-87c9-4b96-8a09-e4fd779afae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:52.756453 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0cbd6a-87c9-4b96-8a09-e4fd779afae4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:52.756693 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0cbd6a-87c9-4b96-8a09-e4fd779afae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:52.757042 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0cbd6a-87c9-4b96-8a09-e4fd779afae4 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:52.758840 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0cbd6a-87c9-4b96-8a09-e4fd779afae4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:52.758840 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0cbd6a-87c9-4b96-8a09-e4fd779afae4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:52.759955 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0cbd6a-87c9-4b96-8a09-e4fd779afae4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:52.760760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-8cf2608f-920c-4055-9d37-766e3e5e2972 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:52.760760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:52.760760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:52.760760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:52.760760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:52.760760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:52.760760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:52.760760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:52.760760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:52.760760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:52.760760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:52.760760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:52.760760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:15:52.760760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:15:52.760760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:15:52.762561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:15:52.762561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:52.762561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:52.762561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:52.762561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:52.762561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:52.762561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:52.762561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:52.762561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:52.762561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:52.762561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:52.762561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:52.762561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:52.762561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:52.762561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:52.762561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:52.762561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:52.764350 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:15:52.764350 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:15:52.764350 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:15:52.764350 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:15:52.764350 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:15:52.764350 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:15:52.764350 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:52.764350 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:52.764350 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 372/749] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:52.764350 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf2608f-920c-4055-9d37-766e3e5e2972 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:52.802392 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 378/750] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Jun 30 18:15:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:52.803945 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0cbd6a-87c9-4b96-8a09-e4fd779afae4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:52.829195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:52.880336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:52.894665 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf2608f-920c-4055-9d37-766e3e5e2972 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:52.936359 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242dbac7-6f35-4497-97ef-694d28aae68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:52.937081 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242dbac7-6f35-4497-97ef-694d28aae68b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:52.937367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242dbac7-6f35-4497-97ef-694d28aae68b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:52.937693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242dbac7-6f35-4497-97ef-694d28aae68b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:52.948169 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242dbac7-6f35-4497-97ef-694d28aae68b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:52.949700 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242dbac7-6f35-4497-97ef-694d28aae68b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:52.955878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0cbd6a-87c9-4b96-8a09-e4fd779afae4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:52.966288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-242dbac7-6f35-4497-97ef-694d28aae68b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 18:15:52.966288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:52.966288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:52.966288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:52.966288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:52.966288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:52.966288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:52.966288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:52.966288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:52.966288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:52.966288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:52.966288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:52.966288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:15:52.966288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:15:52.966288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:15:52.966288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:15:52.967329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:52.967329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:52.967329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:15:52.967329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:52.967329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:15:52.967329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:52.967329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:52.967329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:52.967329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:52.967329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:52.967329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:52.967329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:52.967329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:52.967329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:52.967329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:52.967329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:52.967329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:52.967329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:52.968272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:52.968272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:52.968272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:15:52.968272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:15:52.968272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:15:52.968272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:15:52.968272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:15:52.968272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:15:52.968272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 18:15:52.968272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:52.972314 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 373/751] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:15:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:52.973303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242dbac7-6f35-4497-97ef-694d28aae68b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:53.013561 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7e0e7-0f29-43ce-8a0d-cec72472576c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:53.014933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7e0e7-0f29-43ce-8a0d-cec72472576c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:53.015446 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7e0e7-0f29-43ce-8a0d-cec72472576c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:53.016495 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7e0e7-0f29-43ce-8a0d-cec72472576c None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:53.017892 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7e0e7-0f29-43ce-8a0d-cec72472576c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:53.018455 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7e0e7-0f29-43ce-8a0d-cec72472576c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:53.020959 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7e0e7-0f29-43ce-8a0d-cec72472576c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:53.050816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 379/752] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:15:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:53.051843 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7e0e7-0f29-43ce-8a0d-cec72472576c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:53.125592 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242dbac7-6f35-4497-97ef-694d28aae68b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:53.163916 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151233b6-0bfb-4f3b-9d86-d9d14eb9592d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:53.164574 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151233b6-0bfb-4f3b-9d86-d9d14eb9592d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:53.164861 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151233b6-0bfb-4f3b-9d86-d9d14eb9592d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:53.165140 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151233b6-0bfb-4f3b-9d86-d9d14eb9592d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:53.180328 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:53.189435 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:53.212039 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7e0e7-0f29-43ce-8a0d-cec72472576c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:53.239818 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151233b6-0bfb-4f3b-9d86-d9d14eb9592d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:53.241386 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151233b6-0bfb-4f3b-9d86-d9d14eb9592d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:53.253879 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29b976-1be0-4112-8ea7-b98754f2fbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:53.254839 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29b976-1be0-4112-8ea7-b98754f2fbd9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:53.255286 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29b976-1be0-4112-8ea7-b98754f2fbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:53.256642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29b976-1be0-4112-8ea7-b98754f2fbd9 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:53.258060 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29b976-1be0-4112-8ea7-b98754f2fbd9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:53.258534 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29b976-1be0-4112-8ea7-b98754f2fbd9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:53.260365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29b976-1be0-4112-8ea7-b98754f2fbd9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:53.286123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 380/753] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Jun 30 18:15:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:53.288090 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29b976-1be0-4112-8ea7-b98754f2fbd9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:53.357653 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 374/754] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:15:52 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 18:15:53.358840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151233b6-0bfb-4f3b-9d86-d9d14eb9592d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:53.414117 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29b976-1be0-4112-8ea7-b98754f2fbd9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:53.421836 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:53.467523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01826b17-a5cb-41d5-90db-1f55e59e07a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:53.468784 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01826b17-a5cb-41d5-90db-1f55e59e07a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:53.469339 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01826b17-a5cb-41d5-90db-1f55e59e07a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:53.469863 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01826b17-a5cb-41d5-90db-1f55e59e07a4 None admin] PATH_INFO: `/v3/projects/8c7e1ee12aa14b7ab13e07822826b25b/users/b8751a4e92174d18b7b138392d2579a1/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:53.472682 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01826b17-a5cb-41d5-90db-1f55e59e07a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c7e1ee12aa14b7ab13e07822826b25b, user_id=b8751a4e92174d18b7b138392d2579a1, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:53.475129 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01826b17-a5cb-41d5-90db-1f55e59e07a4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:53.478281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:53.482928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151233b6-0bfb-4f3b-9d86-d9d14eb9592d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:53.495205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 381/755] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:15:53 2022] PUT /identity/v3/projects/8c7e1ee12aa14b7ab13e07822826b25b/users/b8751a4e92174d18b7b138392d2579a1/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:15:53.496562 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01826b17-a5cb-41d5-90db-1f55e59e07a4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:53.578959 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfc63c-6b05-4347-bf12-64a64f59bc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:53.579701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfc63c-6b05-4347-bf12-64a64f59bc4f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:53.579992 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfc63c-6b05-4347-bf12-64a64f59bc4f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:53.580335 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfc63c-6b05-4347-bf12-64a64f59bc4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:53.632470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01826b17-a5cb-41d5-90db-1f55e59e07a4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:53.677229 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885808e-03de-49e0-b3c9-b50b10d211eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:53.679979 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885808e-03de-49e0-b3c9-b50b10d211eb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:53.680533 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885808e-03de-49e0-b3c9-b50b10d211eb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:53.681117 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885808e-03de-49e0-b3c9-b50b10d211eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:53.695278 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3885808e-03de-49e0-b3c9-b50b10d211eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:53.697187 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3885808e-03de-49e0-b3c9-b50b10d211eb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:53.720421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3885808e-03de-49e0-b3c9-b50b10d211eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:53.720421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:53.720421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:53.720421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:53.720421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:53.720421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:53.720421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:53.720421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:53.720421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:53.720421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:53.720421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:53.720421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:53.720421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:53.720421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:53.720421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:53.721384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:53.721384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:53.721384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:53.721384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:53.721384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:53.721384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:53.721384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:53.721384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:53.721384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:53.721384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:53.721384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:53.721384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:53.721384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:53.721384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:53.721384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:53.721384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:53.721384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:53.722328 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:53.722328 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:53.722328 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:53.722328 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:53.725344 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 382/756] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:53.732631 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885808e-03de-49e0-b3c9-b50b10d211eb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:53.765264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecfc63c-6b05-4347-bf12-64a64f59bc4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:53.768253 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecfc63c-6b05-4347-bf12-64a64f59bc4f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:53.796959 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:53.845889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:53.874332 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885808e-03de-49e0-b3c9-b50b10d211eb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:53.906409 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 375/757] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:15:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 18:15:53.912772 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfc63c-6b05-4347-bf12-64a64f59bc4f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:53.924240 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0bc83b-1e89-467e-b248-0987c88537fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:53.925067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0bc83b-1e89-467e-b248-0987c88537fe None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:53.925493 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0bc83b-1e89-467e-b248-0987c88537fe None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:53.925938 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0bc83b-1e89-467e-b248-0987c88537fe None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:53.928143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0bc83b-1e89-467e-b248-0987c88537fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:53.929132 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0bc83b-1e89-467e-b248-0987c88537fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:53.930557 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0bc83b-1e89-467e-b248-0987c88537fe None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:53.952695 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 383/758] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:15:53.960708 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0bc83b-1e89-467e-b248-0987c88537fe None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:54.110807 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfc63c-6b05-4347-bf12-64a64f59bc4f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:54.169608 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806eb4c5-66dd-4cb9-9e0e-4b42ab934463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:54.169608 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806eb4c5-66dd-4cb9-9e0e-4b42ab934463 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:54.169608 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806eb4c5-66dd-4cb9-9e0e-4b42ab934463 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:54.169608 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806eb4c5-66dd-4cb9-9e0e-4b42ab934463 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:54.197924 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806eb4c5-66dd-4cb9-9e0e-4b42ab934463 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:54.199544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806eb4c5-66dd-4cb9-9e0e-4b42ab934463 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:54.230598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0bc83b-1e89-467e-b248-0987c88537fe None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:54.230946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-806eb4c5-66dd-4cb9-9e0e-4b42ab934463 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:54.230946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:54.230946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:54.230946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:54.230946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:54.230946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:54.230946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:54.230946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:54.230946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:54.230946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:54.230946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:54.230946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:54.230946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:54.230946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:54.230946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:54.231841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:54.231841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:54.231841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:54.231841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:54.231841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:54.231841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:54.231841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:54.231841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:54.231841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:54.231841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:54.231841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:54.231841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:54.231841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:54.231841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:54.231841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:54.231841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:54.231841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:54.232723 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:54.232723 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:54.232723 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:54.232723 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:54.242282 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:54.243292 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 376/759] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:54.248411 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806eb4c5-66dd-4cb9-9e0e-4b42ab934463 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:54.257305 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:54.304848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97081bc-ef17-44af-864d-f261e395299e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:54.306409 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97081bc-ef17-44af-864d-f261e395299e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:54.306932 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97081bc-ef17-44af-864d-f261e395299e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:54.307436 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97081bc-ef17-44af-864d-f261e395299e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:54.320016 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97081bc-ef17-44af-864d-f261e395299e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:54.323222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97081bc-ef17-44af-864d-f261e395299e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:54.355317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-c97081bc-ef17-44af-864d-f261e395299e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:54.355317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:54.355317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:54.355317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:54.355317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:54.355317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:54.355317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:54.355317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:54.355317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:54.355317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:54.355317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:54.355317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:54.355317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:15:54.355317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:15:54.355317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:15:54.356292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:15:54.356292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:54.356292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:54.356292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:54.356292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:54.356292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:54.356292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:54.356292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:54.356292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:54.356292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:54.356292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:54.356292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:54.356292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:54.356292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:54.356292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:54.356292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:54.356292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:54.357180 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:15:54.357180 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:15:54.357180 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:15:54.357180 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:15:54.357180 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:15:54.357180 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:15:54.357180 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:54.357180 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:54.358172 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 384/760] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:15:54.366584 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97081bc-ef17-44af-864d-f261e395299e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:54.476583 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806eb4c5-66dd-4cb9-9e0e-4b42ab934463 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:54.527668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643f7de-2d8b-4525-ad1f-e875717ab53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:54.530549 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643f7de-2d8b-4525-ad1f-e875717ab53b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:54.531416 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643f7de-2d8b-4525-ad1f-e875717ab53b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:54.533289 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643f7de-2d8b-4525-ad1f-e875717ab53b None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:54.534527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643f7de-2d8b-4525-ad1f-e875717ab53b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:54.535527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643f7de-2d8b-4525-ad1f-e875717ab53b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:54.538484 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643f7de-2d8b-4525-ad1f-e875717ab53b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:54.563767 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 377/761] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:15:54.571798 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643f7de-2d8b-4525-ad1f-e875717ab53b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:54.655302 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97081bc-ef17-44af-864d-f261e395299e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:54.661997 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:54.711185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5092f4d4-d752-4356-bce1-dbb40a56e8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:54.712611 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092f4d4-d752-4356-bce1-dbb40a56e8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:54.713431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092f4d4-d752-4356-bce1-dbb40a56e8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:54.713962 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092f4d4-d752-4356-bce1-dbb40a56e8c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:54.715655 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5092f4d4-d752-4356-bce1-dbb40a56e8c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:54.716233 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5092f4d4-d752-4356-bce1-dbb40a56e8c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:54.717281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643f7de-2d8b-4525-ad1f-e875717ab53b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:54.718308 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5092f4d4-d752-4356-bce1-dbb40a56e8c3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:54.737933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 385/762] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:15:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:54.740397 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e742c22-581a-45d5-adab-506de51d1292 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:54.741215 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e742c22-581a-45d5-adab-506de51d1292 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:54.741748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e742c22-581a-45d5-adab-506de51d1292 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:54.744674 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 386/763] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:54.745831 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e742c22-581a-45d5-adab-506de51d1292 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:54.769243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b478e8-0d00-4934-a5ab-ae70a25457d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:54.770138 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b478e8-0d00-4934-a5ab-ae70a25457d0 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:54.770457 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b478e8-0d00-4934-a5ab-ae70a25457d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:54.770924 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b478e8-0d00-4934-a5ab-ae70a25457d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:54.779623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b478e8-0d00-4934-a5ab-ae70a25457d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:54.781533 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b478e8-0d00-4934-a5ab-ae70a25457d0 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:54.784032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:54.801432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-65b478e8-0d00-4934-a5ab-ae70a25457d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:54.801432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:54.801432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:54.801432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:54.801432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:54.801432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:54.801432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:54.801432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:54.801432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:54.801432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:54.801432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:54.801432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:54.801432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:15:54.801432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:15:54.801432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:15:54.803082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:15:54.803082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:54.803082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:54.803082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:54.803082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:54.803082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:54.803082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:54.803082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:54.803082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:54.803082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:54.803082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:54.803082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:54.803082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:54.803082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:54.803082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:54.803082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:54.803082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:54.804438 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:15:54.804438 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:15:54.804438 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:15:54.804438 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:15:54.804438 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:15:54.804438 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:15:54.804438 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:54.804438 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:54.808068 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 378/764] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:54.809836 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63909332-a35a-4e48-9b68-fa91a627cfa1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:54.810165 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63909332-a35a-4e48-9b68-fa91a627cfa1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:54.810389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63909332-a35a-4e48-9b68-fa91a627cfa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:54.829531 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-63909332-a35a-4e48-9b68-fa91a627cfa1 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:54.830715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-63909332-a35a-4e48-9b68-fa91a627cfa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:54.877669 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e742c22-581a-45d5-adab-506de51d1292 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:54.925088 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 379/765] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:15:54.926119 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63909332-a35a-4e48-9b68-fa91a627cfa1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:54.946293 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e67f20-16a8-4b45-b90f-b40c19483252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:54.946960 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e67f20-16a8-4b45-b90f-b40c19483252 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:54.947284 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e67f20-16a8-4b45-b90f-b40c19483252 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:54.947585 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e67f20-16a8-4b45-b90f-b40c19483252 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:54.948373 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e67f20-16a8-4b45-b90f-b40c19483252 None admin] RBAC: Adding query filter params (role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d, scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b, user.id=4ddcf6306e704fe9ad5ed8779178ae67) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:54.948692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e67f20-16a8-4b45-b90f-b40c19483252 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:54.950217 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e67f20-16a8-4b45-b90f-b40c19483252 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:54.957181 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 387/766] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:15:54 2022] GET /identity/v3/role_assignments?user.id=4ddcf6306e704fe9ad5ed8779178ae67&scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b&role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:54.958924 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996a908-ee81-487b-abda-7977f68d7e69 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:54.959743 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996a908-ee81-487b-abda-7977f68d7e69 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:54.959743 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996a908-ee81-487b-abda-7977f68d7e69 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:54.961308 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 388/767] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:54.968669 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39128d3c-a10e-45a7-a92d-174950a3acc6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:54.969084 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39128d3c-a10e-45a7-a92d-174950a3acc6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:54.969258 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39128d3c-a10e-45a7-a92d-174950a3acc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:55.019742 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-39128d3c-a10e-45a7-a92d-174950a3acc6 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:55.022566 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-39128d3c-a10e-45a7-a92d-174950a3acc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:55.039407 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:55.111355 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63909332-a35a-4e48-9b68-fa91a627cfa1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:55.118420 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 389/768] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:15:55.119362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39128d3c-a10e-45a7-a92d-174950a3acc6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:55.144068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:55.173204 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e5b6b-57d5-448b-84dd-6062d7dc12bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:55.174396 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e5b6b-57d5-448b-84dd-6062d7dc12bc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:55.174941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e5b6b-57d5-448b-84dd-6062d7dc12bc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:55.175302 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e5b6b-57d5-448b-84dd-6062d7dc12bc None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:55.176148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e5b6b-57d5-448b-84dd-6062d7dc12bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:55.176464 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e5b6b-57d5-448b-84dd-6062d7dc12bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:55.178572 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e5b6b-57d5-448b-84dd-6062d7dc12bc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:55.201764 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 380/769] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:15:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:55.203247 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e5b6b-57d5-448b-84dd-6062d7dc12bc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:55.293858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39128d3c-a10e-45a7-a92d-174950a3acc6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:55.377753 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f715e-61b6-42b5-a617-25884aa03dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:55.379094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f715e-61b6-42b5-a617-25884aa03dde None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:55.379208 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e5b6b-57d5-448b-84dd-6062d7dc12bc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:55.379738 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f715e-61b6-42b5-a617-25884aa03dde None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:55.380262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f715e-61b6-42b5-a617-25884aa03dde None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:55.383551 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f715e-61b6-42b5-a617-25884aa03dde None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:55.386110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f715e-61b6-42b5-a617-25884aa03dde None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:55.398335 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:55.420356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-8c8f715e-61b6-42b5-a617-25884aa03dde None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 18:15:55.420356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:55.420356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:15:55.420356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:15:55.420356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:15:55.420356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:15:55.420356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:15:55.420356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:15:55.420356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 18:15:55.420356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:55.420356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:15:55.420356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:55.420356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:55.420356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:55.420356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:55.420356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:55.422741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:55.422741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:55.422741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:55.422741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:55.422741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:55.422741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:55.422741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:55.422741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:15:55.422741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:15:55.422741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:15:55.422741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:15:55.422741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:55.422741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:55.422741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:55.422741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:55.422741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:55.424450 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:55.424450 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:55.424450 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:55.424450 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:55.424450 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:55.424450 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:55.424450 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:55.424450 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:55.424450 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:55.424450 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:55.424450 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:55.424450 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:15:55.424450 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:15:55.424450 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 18:15:55.424450 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:55.424450 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 390/770] 10.222.0.33 () {56 vars in 1129 bytes} [Thu Jun 30 18:15:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:55.429846 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f715e-61b6-42b5-a617-25884aa03dde None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:55.434908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec22471-5ea2-48b8-970b-690659f4dddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:55.436736 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec22471-5ea2-48b8-970b-690659f4dddb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:55.437620 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec22471-5ea2-48b8-970b-690659f4dddb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:55.439237 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec22471-5ea2-48b8-970b-690659f4dddb None admin] PATH_INFO: `/v3/services/network` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:55.441380 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec22471-5ea2-48b8-970b-690659f4dddb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:55.445675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec22471-5ea2-48b8-970b-690659f4dddb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:55.471281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-6ec22471-5ea2-48b8-970b-690659f4dddb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 18:15:55.471281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:55.471281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:15:55.471281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:15:55.471281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:15:55.471281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:15:55.471281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:15:55.471281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:15:55.471281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 18:15:55.471281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:55.471281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:15:55.471281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:55.471281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:55.471281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:55.471281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:55.471281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:55.471281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:55.472349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:55.472349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:55.472349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:55.472349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:55.472349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:55.472349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:55.472349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:15:55.472349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:15:55.472349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:15:55.472349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:15:55.472349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:55.472349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:55.472349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:55.472349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:55.472349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:55.472349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:55.473202 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:55.473202 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:55.473202 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:55.473202 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:55.473202 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:55.473202 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:55.473202 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:55.473202 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:55.473202 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:55.473202 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:55.473202 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:15:55.473202 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:15:55.473202 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 18:15:55.473202 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:55.477826 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 381/771] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:55 2022] GET /identity/v3/services/network => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:55.480902 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec22471-5ea2-48b8-970b-690659f4dddb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:55.571029 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:55.590567 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f715e-61b6-42b5-a617-25884aa03dde None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:55.645845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89005de2-e391-42cf-a36c-3d4685647818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:55.646716 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec22471-5ea2-48b8-970b-690659f4dddb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:55.647244 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89005de2-e391-42cf-a36c-3d4685647818 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:55.647764 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89005de2-e391-42cf-a36c-3d4685647818 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:55.648283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89005de2-e391-42cf-a36c-3d4685647818 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:55.650315 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89005de2-e391-42cf-a36c-3d4685647818 None admin] RBAC: Adding query filter params (role.id=0ef3687af20a443281e3cfdceca3597d, scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b, user.id=4bd521b9e93f4b3b8af64d46f36c019e) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:55.652291 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89005de2-e391-42cf-a36c-3d4685647818 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:55.655223 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89005de2-e391-42cf-a36c-3d4685647818 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:55.670164 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 391/772] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:15:55 2022] GET /identity/v3/role_assignments?user.id=4bd521b9e93f4b3b8af64d46f36c019e&scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b&role.id=0ef3687af20a443281e3cfdceca3597d => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:55.672682 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89005de2-e391-42cf-a36c-3d4685647818 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:55.719202 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b071917-d6d4-4bcf-b332-7793cc66bcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:55.720288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b071917-d6d4-4bcf-b332-7793cc66bcd5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:55.720785 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b071917-d6d4-4bcf-b332-7793cc66bcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:55.721290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b071917-d6d4-4bcf-b332-7793cc66bcd5 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:55.722303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b071917-d6d4-4bcf-b332-7793cc66bcd5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:55.722838 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b071917-d6d4-4bcf-b332-7793cc66bcd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:55.725110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b071917-d6d4-4bcf-b332-7793cc66bcd5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:55.737689 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 382/773] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Jun 30 18:15:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:55.743523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b071917-d6d4-4bcf-b332-7793cc66bcd5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:55.845989 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:55.925562 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89005de2-e391-42cf-a36c-3d4685647818 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:55.979236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b071917-d6d4-4bcf-b332-7793cc66bcd5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:55.996103 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c43a9-c433-4b5e-95c6-f0fec1d45015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:55.997085 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c43a9-c433-4b5e-95c6-f0fec1d45015 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:55.997247 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c43a9-c433-4b5e-95c6-f0fec1d45015 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:55.997479 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c43a9-c433-4b5e-95c6-f0fec1d45015 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:55.998235 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c43a9-c433-4b5e-95c6-f0fec1d45015 None admin] RBAC: Adding query filter params (name=network) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:55.998551 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c43a9-c433-4b5e-95c6-f0fec1d45015 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:55.999982 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c43a9-c433-4b5e-95c6-f0fec1d45015 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:56.010570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 392/774] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 30 18:15:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:56.018702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c43a9-c433-4b5e-95c6-f0fec1d45015 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:56.045163 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6436b9-536f-4cce-b8c5-3f7b4f4b6d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:56.047019 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6436b9-536f-4cce-b8c5-3f7b4f4b6d54 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:56.047757 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6436b9-536f-4cce-b8c5-3f7b4f4b6d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:56.048528 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6436b9-536f-4cce-b8c5-3f7b4f4b6d54 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:56.050200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6436b9-536f-4cce-b8c5-3f7b4f4b6d54 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:56.051020 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6436b9-536f-4cce-b8c5-3f7b4f4b6d54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:56.054562 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6436b9-536f-4cce-b8c5-3f7b4f4b6d54 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:56.065491 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 383/775] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Jun 30 18:15:55 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:56.078362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:56.081416 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6436b9-536f-4cce-b8c5-3f7b4f4b6d54 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:56.185269 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c43a9-c433-4b5e-95c6-f0fec1d45015 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:56.233673 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75183e7-cbb6-493d-a2ec-89fe628298c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:56.234368 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75183e7-cbb6-493d-a2ec-89fe628298c7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:56.234652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75183e7-cbb6-493d-a2ec-89fe628298c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:56.234928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75183e7-cbb6-493d-a2ec-89fe628298c7 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:56.235664 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75183e7-cbb6-493d-a2ec-89fe628298c7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:56.235974 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75183e7-cbb6-493d-a2ec-89fe628298c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:56.237397 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75183e7-cbb6-493d-a2ec-89fe628298c7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:56.253042 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 393/776] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 30 18:15:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:56.258933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75183e7-cbb6-493d-a2ec-89fe628298c7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:56.297137 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6436b9-536f-4cce-b8c5-3f7b4f4b6d54 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:56.312748 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:56.371671 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ecf4d5-469f-4904-ae0d-519809237ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:56.373248 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ecf4d5-469f-4904-ae0d-519809237ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:56.385658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ecf4d5-469f-4904-ae0d-519809237ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:56.385658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ecf4d5-469f-4904-ae0d-519809237ec1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:56.385658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ecf4d5-469f-4904-ae0d-519809237ec1 None admin] RBAC: Adding query filter params (interface=public, service_id=16a0b8df5b78429a953977f86953fc81, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:56.385658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ecf4d5-469f-4904-ae0d-519809237ec1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:56.385658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ecf4d5-469f-4904-ae0d-519809237ec1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:56.393626 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 384/777] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 30 18:15:56 2022] GET /identity/v3/endpoints?service_id=16a0b8df5b78429a953977f86953fc81&interface=public®ion_id=RegionOne => generated 194 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:56.393626 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2744a65-64cd-4fbb-8bc1-ad5fe63e7b1a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:56.393626 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2744a65-64cd-4fbb-8bc1-ad5fe63e7b1a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:56.393626 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2744a65-64cd-4fbb-8bc1-ad5fe63e7b1a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:56.394586 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 385/778] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:56.403294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cf7bb-b167-49f6-b642-7074c060855f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:56.403574 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cf7bb-b167-49f6-b642-7074c060855f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:56.403760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cf7bb-b167-49f6-b642-7074c060855f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:56.424851 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-527cf7bb-b167-49f6-b642-7074c060855f None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:56.426125 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-527cf7bb-b167-49f6-b642-7074c060855f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:56.461905 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:56.510965 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75183e7-cbb6-493d-a2ec-89fe628298c7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:56.517677 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 386/779] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:15:56.520325 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527cf7bb-b167-49f6-b642-7074c060855f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:56.586657 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd8296-ab16-4f4f-adde-646e06335fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:56.587923 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd8296-ab16-4f4f-adde-646e06335fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:56.588448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd8296-ab16-4f4f-adde-646e06335fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:56.588960 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd8296-ab16-4f4f-adde-646e06335fa4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:56.590874 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd8296-ab16-4f4f-adde-646e06335fa4 None admin] RBAC: Adding query filter params (interface=public, service_id=1f9ce7250eca4ea5a0bea42b91218b3f, region_id=RegionOne) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:56.591453 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd8296-ab16-4f4f-adde-646e06335fa4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:56.594322 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd8296-ab16-4f4f-adde-646e06335fa4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:56.614100 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 394/780] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 30 18:15:56 2022] GET /identity/v3/endpoints?service_id=1f9ce7250eca4ea5a0bea42b91218b3f&interface=public®ion_id=RegionOne => generated 194 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:15:56.615999 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd8296-ab16-4f4f-adde-646e06335fa4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:56.675919 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:56.682179 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527cf7bb-b167-49f6-b642-7074c060855f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:56.728472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6228f18-cc3a-49eb-8942-0a18b03d4efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:56.730077 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6228f18-cc3a-49eb-8942-0a18b03d4efc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:56.730591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6228f18-cc3a-49eb-8942-0a18b03d4efc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:56.731350 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6228f18-cc3a-49eb-8942-0a18b03d4efc None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:56.732270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6228f18-cc3a-49eb-8942-0a18b03d4efc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:56.733947 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6228f18-cc3a-49eb-8942-0a18b03d4efc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:56.750118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 387/781] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 30 18:15:56 2022] GET /identity/v3/services => generated 772 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:56.750118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6228f18-cc3a-49eb-8942-0a18b03d4efc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:56.851453 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd8296-ab16-4f4f-adde-646e06335fa4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:56.976791 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ea8e0-1d8d-4776-81bf-912a1787a989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:56.978060 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ea8e0-1d8d-4776-81bf-912a1787a989 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:56.978627 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ea8e0-1d8d-4776-81bf-912a1787a989 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:56.979153 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ea8e0-1d8d-4776-81bf-912a1787a989 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:56.991532 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:57.000100 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ea8e0-1d8d-4776-81bf-912a1787a989 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:57.006643 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6228f18-cc3a-49eb-8942-0a18b03d4efc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:57.011138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-057ea8e0-1d8d-4776-81bf-912a1787a989 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:15:57.011138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:57.011138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:57.011138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:57.011138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:57.011138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:57.011138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:57.011138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:57.011138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:57.011138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:57.011138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:57.011138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:57.011138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:57.011138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:57.011138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:57.011138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:57.012232 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:57.012232 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:57.012232 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:57.012232 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:57.012232 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:57.012232 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:57.012232 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:57.012232 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:57.012232 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:57.012232 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:57.012232 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:57.012232 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:57.012232 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:57.012232 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:57.012232 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:57.012232 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:57.013163 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:57.013163 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:57.013163 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:57.013163 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:57.013163 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:57.013163 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:57.013163 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:57.013163 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:57.013163 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:57.013163 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:57.013163 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:15:57.013163 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:57.016251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 395/782] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:15:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:57.032808 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ea8e0-1d8d-4776-81bf-912a1787a989 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:57.068719 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27648edc-ee0e-41a9-abed-e58ef07e1a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:57.069522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27648edc-ee0e-41a9-abed-e58ef07e1a18 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:57.069981 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27648edc-ee0e-41a9-abed-e58ef07e1a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:57.070291 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27648edc-ee0e-41a9-abed-e58ef07e1a18 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:57.071184 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27648edc-ee0e-41a9-abed-e58ef07e1a18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:57.072409 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27648edc-ee0e-41a9-abed-e58ef07e1a18 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:57.078658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 388/783] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 30 18:15:56 2022] GET /identity/v3/services => generated 772 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:57.166359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ea8e0-1d8d-4776-81bf-912a1787a989 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:57.213509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623840e-bcda-49f6-8d98-c02beb2bfd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:57.214225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623840e-bcda-49f6-8d98-c02beb2bfd3f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:57.214512 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623840e-bcda-49f6-8d98-c02beb2bfd3f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:57.214791 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623840e-bcda-49f6-8d98-c02beb2bfd3f None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:57.215558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623840e-bcda-49f6-8d98-c02beb2bfd3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:57.215867 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623840e-bcda-49f6-8d98-c02beb2bfd3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:57.217263 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623840e-bcda-49f6-8d98-c02beb2bfd3f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:57.234135 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 396/784] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:15:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:57.235508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27648edc-ee0e-41a9-abed-e58ef07e1a18 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:57.283733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:57.390614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27648edc-ee0e-41a9-abed-e58ef07e1a18 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:57.445956 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7f84d-f7d3-402f-bf17-296269ca97f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:57.446780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7f84d-f7d3-402f-bf17-296269ca97f2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:57.447068 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7f84d-f7d3-402f-bf17-296269ca97f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:57.447349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7f84d-f7d3-402f-bf17-296269ca97f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:57.460524 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f7f84d-f7d3-402f-bf17-296269ca97f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:57.462063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f7f84d-f7d3-402f-bf17-296269ca97f2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:57.485268 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b9f7f84d-f7d3-402f-bf17-296269ca97f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:57.485268 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:57.485268 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:57.485268 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:57.485268 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:57.485268 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:57.485268 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:57.485268 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:57.485268 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:57.485268 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:57.485268 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:57.485268 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:57.485268 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:57.485268 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:57.485268 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:57.486629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:57.486629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:57.486629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:57.486629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:57.486629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:57.486629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:57.486629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:57.486629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:57.486629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:57.486629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:57.486629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:57.486629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:57.486629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:57.486629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:57.486629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:57.486629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:57.486629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:57.488352 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:57.488352 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:57.488352 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:57.488352 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:57.492297 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 389/785] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:57.493925 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623840e-bcda-49f6-8d98-c02beb2bfd3f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:57.556605 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240e72c-e893-4f7c-86ea-ee5548383d54 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:57.557059 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240e72c-e893-4f7c-86ea-ee5548383d54 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:57.557315 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240e72c-e893-4f7c-86ea-ee5548383d54 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:57.562736 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 390/786] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:15:57.564523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f4f91-8fbf-443a-a72e-2850a418c2da None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:57.564800 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f4f91-8fbf-443a-a72e-2850a418c2da None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:57.565071 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f4f91-8fbf-443a-a72e-2850a418c2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:57.589978 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:57.590693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-578f4f91-8fbf-443a-a72e-2850a418c2da None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:57.592039 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-578f4f91-8fbf-443a-a72e-2850a418c2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:57.660205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623840e-bcda-49f6-8d98-c02beb2bfd3f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:57.720658 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d048fd-743b-4b60-9e1d-2e3a8c43b7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:57.721415 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d048fd-743b-4b60-9e1d-2e3a8c43b7aa None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:57.721762 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d048fd-743b-4b60-9e1d-2e3a8c43b7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:57.722053 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d048fd-743b-4b60-9e1d-2e3a8c43b7aa None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:57.722830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d048fd-743b-4b60-9e1d-2e3a8c43b7aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:57.723147 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d048fd-743b-4b60-9e1d-2e3a8c43b7aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:57.724810 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d048fd-743b-4b60-9e1d-2e3a8c43b7aa None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:57.744083 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 397/787] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:57.745495 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d048fd-743b-4b60-9e1d-2e3a8c43b7aa None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:57.751253 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 391/788] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:15:57.794680 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f4f91-8fbf-443a-a72e-2850a418c2da None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:57.815692 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:57.912561 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d048fd-743b-4b60-9e1d-2e3a8c43b7aa None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:57.991935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:58.000535 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85683aa-5d63-46f7-9b3d-9818c6825c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:58.001282 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85683aa-5d63-46f7-9b3d-9818c6825c6e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:58.001595 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85683aa-5d63-46f7-9b3d-9818c6825c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:58.001898 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85683aa-5d63-46f7-9b3d-9818c6825c6e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:58.014599 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85683aa-5d63-46f7-9b3d-9818c6825c6e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:58.016095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85683aa-5d63-46f7-9b3d-9818c6825c6e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:58.037535 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f4f91-8fbf-443a-a72e-2850a418c2da None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:58.047968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b85683aa-5d63-46f7-9b3d-9818c6825c6e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 18:15:58.047968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:58.047968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:58.047968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:58.047968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:58.047968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:58.047968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:58.047968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:58.047968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:58.047968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:58.047968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:58.047968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:58.047968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:15:58.047968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:15:58.047968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:15:58.047968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:15:58.048989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:58.048989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:58.048989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:15:58.048989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:58.048989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:15:58.048989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:58.048989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:58.048989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:58.048989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:58.048989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:58.048989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:58.048989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:58.048989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:58.048989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:58.048989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:58.048989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:58.048989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:58.048989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:58.049987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:58.049987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:58.049987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:15:58.049987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:15:58.049987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:15:58.049987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:15:58.049987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:15:58.049987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:15:58.049987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 18:15:58.049987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:58.049987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 398/789] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 30 18:15:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:15:58.070344 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85683aa-5d63-46f7-9b3d-9818c6825c6e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:58.125167 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1428505-c444-49fc-be94-53fb3407feaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:58.126019 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1428505-c444-49fc-be94-53fb3407feaf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:58.126311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1428505-c444-49fc-be94-53fb3407feaf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:58.126590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1428505-c444-49fc-be94-53fb3407feaf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:58.154169 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1428505-c444-49fc-be94-53fb3407feaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:58.163099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d1428505-c444-49fc-be94-53fb3407feaf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:58.163099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:58.163099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:58.163099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:58.163099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:58.163099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:58.163099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:58.163099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:58.163099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:58.163099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:58.163099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:58.163099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:58.163099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:58.163099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:58.163099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:58.163099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:58.164368 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:58.164368 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:58.164368 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:58.164368 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:58.164368 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:58.164368 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:58.164368 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:58.164368 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:58.164368 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:58.164368 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:58.164368 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:58.164368 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:58.164368 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:58.164368 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:58.164368 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:58.164368 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:58.165381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:58.165381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:58.165381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:58.165381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:58.165381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:58.165381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:58.165381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:58.165381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:58.165381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:58.165381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:58.165381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:15:58.165381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:58.165381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 392/790] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 30 18:15:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:58.173021 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1428505-c444-49fc-be94-53fb3407feaf None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:58.232899 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85683aa-5d63-46f7-9b3d-9818c6825c6e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:58.276171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab09dc9-95fd-4770-af53-fe2c35ed0c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:58.277173 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab09dc9-95fd-4770-af53-fe2c35ed0c71 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:58.277708 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab09dc9-95fd-4770-af53-fe2c35ed0c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:58.280415 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab09dc9-95fd-4770-af53-fe2c35ed0c71 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:58.281554 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab09dc9-95fd-4770-af53-fe2c35ed0c71 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:58.282117 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab09dc9-95fd-4770-af53-fe2c35ed0c71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:58.283790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab09dc9-95fd-4770-af53-fe2c35ed0c71 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:58.310428 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 399/791] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 30 18:15:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:58.320368 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab09dc9-95fd-4770-af53-fe2c35ed0c71 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:58.366144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:58.413850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:58.447979 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1428505-c444-49fc-be94-53fb3407feaf None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:58.500754 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab09dc9-95fd-4770-af53-fe2c35ed0c71 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:58.514909 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cefdf7-348f-456a-84cc-7918e6187654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:58.515938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cefdf7-348f-456a-84cc-7918e6187654 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:58.516483 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cefdf7-348f-456a-84cc-7918e6187654 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:58.516971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cefdf7-348f-456a-84cc-7918e6187654 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:58.518152 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cefdf7-348f-456a-84cc-7918e6187654 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:58.518470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cefdf7-348f-456a-84cc-7918e6187654 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:58.519881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cefdf7-348f-456a-84cc-7918e6187654 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:58.530810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 393/792] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 30 18:15:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:15:58.531701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee859b23-1e65-43c2-b268-a85cfc7aaecd None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:58.532044 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee859b23-1e65-43c2-b268-a85cfc7aaecd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:58.532319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee859b23-1e65-43c2-b268-a85cfc7aaecd None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:58.538220 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 394/793] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:58.538220 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee859b23-1e65-43c2-b268-a85cfc7aaecd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:58.550376 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195813c-f461-4e92-94e1-8ea7d7c8c931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:58.551035 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195813c-f461-4e92-94e1-8ea7d7c8c931 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:58.551325 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195813c-f461-4e92-94e1-8ea7d7c8c931 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:58.551603 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195813c-f461-4e92-94e1-8ea7d7c8c931 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:58.650844 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195813c-f461-4e92-94e1-8ea7d7c8c931 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:58.652404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195813c-f461-4e92-94e1-8ea7d7c8c931 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:58.678647 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee859b23-1e65-43c2-b268-a85cfc7aaecd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:58.711076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:58.724039 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:58.731649 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab25512-0c60-4354-b90b-a9f57d469931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:58.733308 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab25512-0c60-4354-b90b-a9f57d469931 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:58.733962 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab25512-0c60-4354-b90b-a9f57d469931 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:58.734486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab25512-0c60-4354-b90b-a9f57d469931 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:58.749203 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab25512-0c60-4354-b90b-a9f57d469931 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:58.751983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab25512-0c60-4354-b90b-a9f57d469931 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:58.774509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4ab25512-0c60-4354-b90b-a9f57d469931 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:58.774509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:58.774509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:58.774509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:58.774509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:58.774509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:58.774509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:58.774509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:58.774509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:58.774509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:58.774509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:58.774509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:58.774509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:58.774509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:58.774509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:58.775469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:58.775469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:58.775469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:58.775469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:58.775469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:58.775469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:58.775469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:58.775469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:58.775469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:58.775469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:58.775469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:58.775469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:58.775469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:58.775469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:58.775469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:58.775469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:58.775469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:58.776379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:58.776379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:58.776379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:58.776379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:58.778936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 395/794] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:58.780619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77ca65-ec66-4c17-b443-44fa712ce88b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:58.781475 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77ca65-ec66-4c17-b443-44fa712ce88b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:58.782058 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77ca65-ec66-4c17-b443-44fa712ce88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:58.794667 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 400/795] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:15:58 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 18:15:58.796449 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195813c-f461-4e92-94e1-8ea7d7c8c931 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:58.802956 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7a77ca65-ec66-4c17-b443-44fa712ce88b None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:58.804497 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7a77ca65-ec66-4c17-b443-44fa712ce88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:58.908640 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 396/796] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:15:58.909550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77ca65-ec66-4c17-b443-44fa712ce88b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:58.952713 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195813c-f461-4e92-94e1-8ea7d7c8c931 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:58.995359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c49e2e-aca5-4684-83a8-0f53dd6ed4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:58.996102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c49e2e-aca5-4684-83a8-0f53dd6ed4e7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:58.996398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c49e2e-aca5-4684-83a8-0f53dd6ed4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:58.996681 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c49e2e-aca5-4684-83a8-0f53dd6ed4e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:58.997526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c49e2e-aca5-4684-83a8-0f53dd6ed4e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:58.997870 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c49e2e-aca5-4684-83a8-0f53dd6ed4e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:58.999473 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c49e2e-aca5-4684-83a8-0f53dd6ed4e7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:59.008970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:59.014975 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 401/797] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:59.015864 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c49e2e-aca5-4684-83a8-0f53dd6ed4e7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:59.056434 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77ca65-ec66-4c17-b443-44fa712ce88b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:59.080188 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:59.119285 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5912df-b330-4221-a8f4-4116043e9401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:59.120714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5912df-b330-4221-a8f4-4116043e9401 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:59.121204 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5912df-b330-4221-a8f4-4116043e9401 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:59.121726 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5912df-b330-4221-a8f4-4116043e9401 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:59.144431 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5912df-b330-4221-a8f4-4116043e9401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:59.148025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5912df-b330-4221-a8f4-4116043e9401 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:59.173619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-4b5912df-b330-4221-a8f4-4116043e9401 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:59.173619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:59.173619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:59.173619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:59.173619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:59.173619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:59.173619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:59.173619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:59.173619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:59.173619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:59.173619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:59.173619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:59.173619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:15:59.173619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:15:59.173619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:15:59.174588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:15:59.174588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:59.174588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:59.174588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:59.174588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:59.174588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:59.174588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:59.174588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:59.174588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:59.174588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:59.174588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:59.174588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:59.174588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:59.174588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:59.174588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:59.174588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:59.174588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:59.175471 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:15:59.175471 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:15:59.175471 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:15:59.175471 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:59.175471 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 397/798] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:59.176201 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5912df-b330-4221-a8f4-4116043e9401 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:59.179252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c49e2e-aca5-4684-83a8-0f53dd6ed4e7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:59.223037 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4736c9d-f82f-4e18-9776-49d6b0649258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:59.224004 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4736c9d-f82f-4e18-9776-49d6b0649258 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:59.224867 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4736c9d-f82f-4e18-9776-49d6b0649258 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:59.225163 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4736c9d-f82f-4e18-9776-49d6b0649258 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:59.238093 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4736c9d-f82f-4e18-9776-49d6b0649258 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:15:59.246984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b4736c9d-f82f-4e18-9776-49d6b0649258 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:15:59.246984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:59.246984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:59.246984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:59.246984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:59.246984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:59.246984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:59.246984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:59.246984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:59.246984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:59.246984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:59.246984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:59.246984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:15:59.246984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:15:59.246984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:15:59.246984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:15:59.248051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:15:59.248051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:15:59.248051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:15:59.248051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:15:59.248051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:59.248051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:59.248051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:59.248051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:59.248051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:59.248051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:59.248051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:59.248051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:59.248051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:59.248051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:59.248051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:59.248051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:59.248873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:59.248873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:59.248873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:59.248873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:59.248873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:15:59.248873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:15:59.248873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:15:59.248873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:15:59.248873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:15:59.248873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:15:59.248873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:15:59.248873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:15:59.248873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 402/799] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:15:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:59.249688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4736c9d-f82f-4e18-9776-49d6b0649258 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:59.299115 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5912df-b330-4221-a8f4-4116043e9401 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:59.319337 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:59.345704 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca33029-dec4-4252-89c0-608c9f77e11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:59.346533 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca33029-dec4-4252-89c0-608c9f77e11a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:59.346813 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca33029-dec4-4252-89c0-608c9f77e11a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:59.347095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca33029-dec4-4252-89c0-608c9f77e11a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:59.356451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca33029-dec4-4252-89c0-608c9f77e11a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:59.357964 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca33029-dec4-4252-89c0-608c9f77e11a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:59.367148 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4736c9d-f82f-4e18-9776-49d6b0649258 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:59.374928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-dca33029-dec4-4252-89c0-608c9f77e11a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 18:15:59.374928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:59.374928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:59.374928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:59.374928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:59.374928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:59.374928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:59.374928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:59.374928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:59.374928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:59.374928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:59.374928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:59.374928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:15:59.374928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:15:59.374928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:15:59.374928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:15:59.377086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:59.377086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:59.377086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:15:59.377086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:59.377086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:15:59.377086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:15:59.377086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:59.377086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:59.377086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:59.377086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:59.377086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:59.377086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:59.377086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:59.377086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:59.377086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:59.377086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:59.377086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:59.377086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:59.379336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:59.379336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:59.379336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:15:59.379336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:15:59.379336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:15:59.379336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:15:59.379336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:15:59.379336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:15:59.379336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 18:15:59.379336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:59.382222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 398/800] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 30 18:15:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:59.383036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca33029-dec4-4252-89c0-608c9f77e11a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:59.414610 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29564d-0f8e-4a36-845e-50466c1b53d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:59.415483 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29564d-0f8e-4a36-845e-50466c1b53d6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:59.416211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29564d-0f8e-4a36-845e-50466c1b53d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:59.416211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29564d-0f8e-4a36-845e-50466c1b53d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:59.417063 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29564d-0f8e-4a36-845e-50466c1b53d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:59.417396 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29564d-0f8e-4a36-845e-50466c1b53d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:59.419136 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:59.419977 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29564d-0f8e-4a36-845e-50466c1b53d6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:59.442822 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 403/801] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:59.445350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2538a3-0bbf-40ff-b721-955ee854de6d None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:59.445674 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2538a3-0bbf-40ff-b721-955ee854de6d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:59.445962 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2538a3-0bbf-40ff-b721-955ee854de6d None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:59.449622 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 404/802] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:15:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:15:59.449622 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2538a3-0bbf-40ff-b721-955ee854de6d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:59.520599 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca33029-dec4-4252-89c0-608c9f77e11a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:59.566789 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2894b-3b70-4dba-9274-b13c3e85bafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:59.567458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2894b-3b70-4dba-9274-b13c3e85bafe None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:59.567738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2894b-3b70-4dba-9274-b13c3e85bafe None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:59.568098 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2894b-3b70-4dba-9274-b13c3e85bafe None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:59.569280 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2894b-3b70-4dba-9274-b13c3e85bafe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:59.569669 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2894b-3b70-4dba-9274-b13c3e85bafe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:59.571026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2894b-3b70-4dba-9274-b13c3e85bafe None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:59.580413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 399/803] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:15:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:59.581262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2894b-3b70-4dba-9274-b13c3e85bafe None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:59.593993 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2538a3-0bbf-40ff-b721-955ee854de6d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:59.695283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:59.711040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2894b-3b70-4dba-9274-b13c3e85bafe None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:59.752533 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:15:59.752952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284821e-df53-4806-b48d-0ec8a8fc6b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:59.753877 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284821e-df53-4806-b48d-0ec8a8fc6b54 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:59.754188 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284821e-df53-4806-b48d-0ec8a8fc6b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:59.755309 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284821e-df53-4806-b48d-0ec8a8fc6b54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:59.767488 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284821e-df53-4806-b48d-0ec8a8fc6b54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:59.768944 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284821e-df53-4806-b48d-0ec8a8fc6b54 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:59.771421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7192018-1c85-43bf-b46b-b0859a9bbe01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:59.771874 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7192018-1c85-43bf-b46b-b0859a9bbe01 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:59.772169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7192018-1c85-43bf-b46b-b0859a9bbe01 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:59.772431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7192018-1c85-43bf-b46b-b0859a9bbe01 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:59.773412 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7192018-1c85-43bf-b46b-b0859a9bbe01 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:15:59.773755 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7192018-1c85-43bf-b46b-b0859a9bbe01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:59.775201 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7192018-1c85-43bf-b46b-b0859a9bbe01 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:59.784436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-8284821e-df53-4806-b48d-0ec8a8fc6b54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:59.784436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:15:59.784436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:15:59.784436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:15:59.784436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:15:59.784436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:15:59.784436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:15:59.784436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:15:59.784436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:15:59.784436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:15:59.784436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:15:59.784436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:15:59.784436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:15:59.784436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:15:59.784436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:15:59.786786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:15:59.786786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:15:59.786786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:15:59.786786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:15:59.786786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:15:59.786786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:15:59.786786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:15:59.786786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:15:59.786786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:15:59.786786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:15:59.786786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:15:59.786786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:15:59.786786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:15:59.786786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:15:59.786786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:15:59.786786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:15:59.786786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:15:59.789063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:15:59.789063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:15:59.789063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:15:59.789063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:15:59.789063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:15:59.789063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:15:59.789063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:15:59.789063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:15:59.789063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 400/804] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:15:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:15:59.789063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e8097-6132-4659-ba8a-a14c509b17fd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:59.789063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e8097-6132-4659-ba8a-a14c509b17fd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:59.791848 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e8097-6132-4659-ba8a-a14c509b17fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:59.810733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 405/805] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Jun 30 18:15:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:15:59.810733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7192018-1c85-43bf-b46b-b0859a9bbe01 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:59.811006 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1f4e8097-6132-4659-ba8a-a14c509b17fd None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:15:59.811006 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1f4e8097-6132-4659-ba8a-a14c509b17fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:15:59.887802 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 401/806] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:15:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:15:59.888864 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e8097-6132-4659-ba8a-a14c509b17fd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:15:59.926902 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7192018-1c85-43bf-b46b-b0859a9bbe01 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:15:59.968087 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd21bb0-ec72-4e39-af70-871e92ed7475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:15:59.968887 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd21bb0-ec72-4e39-af70-871e92ed7475 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:15:59.969173 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd21bb0-ec72-4e39-af70-871e92ed7475 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:15:59.969454 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd21bb0-ec72-4e39-af70-871e92ed7475 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:15:59.980777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd21bb0-ec72-4e39-af70-871e92ed7475 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:15:59.982329 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd21bb0-ec72-4e39-af70-871e92ed7475 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:15:59.992687 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:00.014024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5fd21bb0-ec72-4e39-af70-871e92ed7475 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:00.014024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:00.014024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:00.014024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:00.014024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:00.014024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:00.014024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:00.014024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:00.014024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:00.014024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:00.014024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:00.014024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:00.014024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:16:00.014024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:16:00.014024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:16:00.015659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:16:00.015659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:00.015659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:00.015659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:00.015659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:00.015659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:00.015659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:00.015659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:00.015659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:00.015659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:00.015659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:00.015659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:00.015659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:00.015659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:00.015659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:00.015659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:00.015659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:00.019160 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:16:00.019160 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:16:00.019160 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:00.019160 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:00.019160 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 406/807] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:15:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:00.019450 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e8097-6132-4659-ba8a-a14c509b17fd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:00.019583 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd21bb0-ec72-4e39-af70-871e92ed7475 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:00.056320 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16775c07-7aac-4c31-895f-5c11a7ff4057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:00.056991 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16775c07-7aac-4c31-895f-5c11a7ff4057 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:00.057561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16775c07-7aac-4c31-895f-5c11a7ff4057 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:00.057561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16775c07-7aac-4c31-895f-5c11a7ff4057 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:00.058382 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16775c07-7aac-4c31-895f-5c11a7ff4057 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:00.058692 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16775c07-7aac-4c31-895f-5c11a7ff4057 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:00.060227 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16775c07-7aac-4c31-895f-5c11a7ff4057 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:00.066015 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:00.075460 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 402/808] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:15:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:00.076400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16775c07-7aac-4c31-895f-5c11a7ff4057 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:00.164562 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd21bb0-ec72-4e39-af70-871e92ed7475 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:00.203874 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446c2d4-fb6f-4a7b-82b0-4c5a8798503c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:00.204679 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446c2d4-fb6f-4a7b-82b0-4c5a8798503c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:00.204963 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446c2d4-fb6f-4a7b-82b0-4c5a8798503c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:00.205247 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446c2d4-fb6f-4a7b-82b0-4c5a8798503c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:00.217444 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446c2d4-fb6f-4a7b-82b0-4c5a8798503c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:00.219114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446c2d4-fb6f-4a7b-82b0-4c5a8798503c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:00.230388 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16775c07-7aac-4c31-895f-5c11a7ff4057 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:00.238084 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-7446c2d4-fb6f-4a7b-82b0-4c5a8798503c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:00.238084 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:00.238084 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:00.238084 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:00.238084 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:00.238084 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:00.238084 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:00.238084 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:00.238084 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:00.238084 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:00.238084 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:00.238084 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:00.238084 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:16:00.238084 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:16:00.238084 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:16:00.239469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:16:00.239469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:00.239469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:00.239469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:00.239469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:00.239469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:00.239469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:00.239469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:00.239469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:00.239469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:00.239469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:00.239469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:00.239469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:00.239469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:00.239469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:00.239469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:00.239469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:00.241382 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:16:00.241382 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:16:00.241382 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:00.241382 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:00.241382 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 407/809] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:00.241856 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7f097-93fd-455f-b380-c2bd36897e65 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:00.242129 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7f097-93fd-455f-b380-c2bd36897e65 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:00.242407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7f097-93fd-455f-b380-c2bd36897e65 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:00.243710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 408/810] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:00.244420 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa7f097-93fd-455f-b380-c2bd36897e65 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:00.276274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a85e2-cb0a-4fd9-87a2-9291ab1ee715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:00.277096 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a85e2-cb0a-4fd9-87a2-9291ab1ee715 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:00.277377 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a85e2-cb0a-4fd9-87a2-9291ab1ee715 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:00.277680 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a85e2-cb0a-4fd9-87a2-9291ab1ee715 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:00.278342 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a85e2-cb0a-4fd9-87a2-9291ab1ee715 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:00.279744 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a85e2-cb0a-4fd9-87a2-9291ab1ee715 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:00.292957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-dc9a85e2-cb0a-4fd9-87a2-9291ab1ee715 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 18:16:00.292957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:00.292957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:16:00.292957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:16:00.292957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:16:00.292957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:16:00.292957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:16:00.292957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:00.292957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 18:16:00.292957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:00.292957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:16:00.292957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:00.292957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:00.292957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:00.292957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:00.292957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:00.294258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:00.294258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:00.294258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:00.294258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:00.294258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:00.294258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:00.294258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:00.294258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:16:00.294258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:16:00.294258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:16:00.294258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:16:00.294258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:00.294258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:00.294258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:00.294258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:00.294258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:00.295408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:00.295408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:00.295408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:00.295408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:00.295408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:00.295408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:00.295408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:00.295408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:00.295408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:00.295408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:00.295408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:00.295408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:16:00.295408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:00.295408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 18:16:00.295408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:00.295408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 403/811] 10.222.0.33 () {56 vars in 1129 bytes} [Thu Jun 30 18:16:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:00.296258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a85e2-cb0a-4fd9-87a2-9291ab1ee715 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:00.298695 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:00.345180 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:00.366569 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa7f097-93fd-455f-b380-c2bd36897e65 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:00.403717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a85e2-cb0a-4fd9-87a2-9291ab1ee715 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:00.404135 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68597dd1-38ac-4d7e-852b-bc04d0e00a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:00.404676 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68597dd1-38ac-4d7e-852b-bc04d0e00a89 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:00.404961 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68597dd1-38ac-4d7e-852b-bc04d0e00a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:00.405245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68597dd1-38ac-4d7e-852b-bc04d0e00a89 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:00.406000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68597dd1-38ac-4d7e-852b-bc04d0e00a89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:00.406312 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68597dd1-38ac-4d7e-852b-bc04d0e00a89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:00.407701 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68597dd1-38ac-4d7e-852b-bc04d0e00a89 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:00.421399 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 409/812] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:00.422221 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68597dd1-38ac-4d7e-852b-bc04d0e00a89 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:00.447118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbd581-4854-41f0-9510-09bdd5887b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:00.447780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbd581-4854-41f0-9510-09bdd5887b98 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:00.448067 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbd581-4854-41f0-9510-09bdd5887b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:00.448346 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbd581-4854-41f0-9510-09bdd5887b98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:00.449184 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbd581-4854-41f0-9510-09bdd5887b98 None admin] RBAC: Adding query filter params (role.id=0ef3687af20a443281e3cfdceca3597d, scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b, user.id=b8751a4e92174d18b7b138392d2579a1) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:00.449498 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbd581-4854-41f0-9510-09bdd5887b98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:00.450974 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbd581-4854-41f0-9510-09bdd5887b98 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:00.456920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 404/813] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:16:00 2022] GET /identity/v3/role_assignments?user.id=b8751a4e92174d18b7b138392d2579a1&scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b&role.id=0ef3687af20a443281e3cfdceca3597d => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:00.458141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a345f-253a-4104-b517-a00a329280c1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:00.458435 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a345f-253a-4104-b517-a00a329280c1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:00.459348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a345f-253a-4104-b517-a00a329280c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:00.481267 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-856a345f-253a-4104-b517-a00a329280c1 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:00.482878 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-856a345f-253a-4104-b517-a00a329280c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:00.553912 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68597dd1-38ac-4d7e-852b-bc04d0e00a89 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:00.574520 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:00.597621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 405/814] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:00.597621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a345f-253a-4104-b517-a00a329280c1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:00.597952 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d57245-a522-4c29-ac06-b6cc469f1b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:00.597952 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d57245-a522-4c29-ac06-b6cc469f1b6b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:00.597952 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d57245-a522-4c29-ac06-b6cc469f1b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:00.597952 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d57245-a522-4c29-ac06-b6cc469f1b6b None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:00.598221 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d57245-a522-4c29-ac06-b6cc469f1b6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:00.598221 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d57245-a522-4c29-ac06-b6cc469f1b6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:00.598221 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d57245-a522-4c29-ac06-b6cc469f1b6b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:00.612547 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 410/815] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:00.613490 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d57245-a522-4c29-ac06-b6cc469f1b6b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:00.616666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:00.734159 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a345f-253a-4104-b517-a00a329280c1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:00.743804 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d57245-a522-4c29-ac06-b6cc469f1b6b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:00.775675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193a15d-f917-47bd-99ee-4ffa08a41015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:00.776425 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193a15d-f917-47bd-99ee-4ffa08a41015 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:00.776839 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193a15d-f917-47bd-99ee-4ffa08a41015 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:00.777311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193a15d-f917-47bd-99ee-4ffa08a41015 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:00.778033 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193a15d-f917-47bd-99ee-4ffa08a41015 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:00.778346 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193a15d-f917-47bd-99ee-4ffa08a41015 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:00.780376 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193a15d-f917-47bd-99ee-4ffa08a41015 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:00.786640 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 406/816] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Jun 30 18:16:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:00.787441 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193a15d-f917-47bd-99ee-4ffa08a41015 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:00.807268 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c1a76-9e2f-4854-b1d9-66697b577faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:00.807897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c1a76-9e2f-4854-b1d9-66697b577faf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:00.808336 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c1a76-9e2f-4854-b1d9-66697b577faf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:00.808513 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c1a76-9e2f-4854-b1d9-66697b577faf None admin] PATH_INFO: `/v3/users/swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:00.824912 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c1a76-9e2f-4854-b1d9-66697b577faf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:00.827732 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c1a76-9e2f-4854-b1d9-66697b577faf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:00.856481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-631c1a76-9e2f-4854-b1d9-66697b577faf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 18:16:00.856481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:00.856481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:00.856481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:00.856481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:00.856481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:00.856481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:00.856481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:00.856481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:00.856481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:00.856481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:00.856481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:00.856481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:16:00.856481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:16:00.856481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:16:00.856481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:16:00.858456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:00.858456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:00.858456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:16:00.858456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:16:00.858456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:16:00.858456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:16:00.858456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:00.858456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:00.858456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:00.858456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:00.858456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:00.858456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:00.858456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:00.858456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:00.858456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:00.858456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:00.858456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:00.858456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:00.860326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:00.860326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:00.860326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:16:00.860326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:16:00.860326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:16:00.860326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:16:00.860326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:16:00.860326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:16:00.860326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 18:16:00.860326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:00.862966 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 411/817] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:16:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:00.863188 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c1a76-9e2f-4854-b1d9-66697b577faf None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:00.875957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:00.939616 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193a15d-f917-47bd-99ee-4ffa08a41015 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:00.958363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:00.994661 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98456ce2-f272-4d62-9f2c-13bedcbd3bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:00.996067 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98456ce2-f272-4d62-9f2c-13bedcbd3bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:00.996693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98456ce2-f272-4d62-9f2c-13bedcbd3bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:00.997431 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98456ce2-f272-4d62-9f2c-13bedcbd3bb9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:01.010166 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98456ce2-f272-4d62-9f2c-13bedcbd3bb9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:01.013388 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98456ce2-f272-4d62-9f2c-13bedcbd3bb9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:01.019172 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c1a76-9e2f-4854-b1d9-66697b577faf None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:01.042780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-98456ce2-f272-4d62-9f2c-13bedcbd3bb9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:16:01.042780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:01.042780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:01.042780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:01.042780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:01.042780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:01.042780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:01.042780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:01.042780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:01.042780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:01.042780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:01.042780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:01.042780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:16:01.042780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:16:01.042780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:16:01.043971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:16:01.043971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:01.043971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:01.043971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:01.043971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:01.043971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:01.043971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:01.043971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:01.043971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:01.043971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:01.043971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:01.043971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:01.043971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:01.043971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:01.043971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:01.043971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:01.043971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:01.045043 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:16:01.045043 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:16:01.045043 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:16:01.045043 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:16:01.045043 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:16:01.045043 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:16:01.045043 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:16:01.045043 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:01.045043 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 407/818] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:16:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:01.046747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98456ce2-f272-4d62-9f2c-13bedcbd3bb9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:01.070207 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343390a-0216-43f6-8dc3-631d5398db63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:01.071313 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343390a-0216-43f6-8dc3-631d5398db63 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:01.071870 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343390a-0216-43f6-8dc3-631d5398db63 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:01.072158 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343390a-0216-43f6-8dc3-631d5398db63 None admin] PATH_INFO: `/v3/services/network` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:01.072841 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343390a-0216-43f6-8dc3-631d5398db63 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:01.074330 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343390a-0216-43f6-8dc3-631d5398db63 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:01.088334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f343390a-0216-43f6-8dc3-631d5398db63 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 18:16:01.088334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:01.088334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:16:01.088334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:16:01.088334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:16:01.088334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:16:01.088334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:16:01.088334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:01.088334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 18:16:01.088334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:01.088334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:16:01.088334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:01.088334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:01.088334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:01.088334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:01.088334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:01.088334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:01.089467 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:01.089467 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:01.089467 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:01.089467 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:01.089467 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:01.089467 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:01.089467 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:16:01.089467 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:16:01.089467 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:16:01.089467 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:16:01.089467 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:01.089467 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:01.089467 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:01.089467 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:01.089467 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:01.089467 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:01.090356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:01.090356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:01.090356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:01.090356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:01.090356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:01.090356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:01.090356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:01.090356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:01.090356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:01.090356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:01.090356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:16:01.090356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:01.090356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 18:16:01.090356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:01.090356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 412/819] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:16:00 2022] GET /identity/v3/services/network => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:01.091155 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343390a-0216-43f6-8dc3-631d5398db63 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:01.208303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98456ce2-f272-4d62-9f2c-13bedcbd3bb9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:01.238318 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:01.242159 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343390a-0216-43f6-8dc3-631d5398db63 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:01.268183 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ee1b7-c51f-49b8-b9be-78f3127d5cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:01.269209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ee1b7-c51f-49b8-b9be-78f3127d5cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:01.269752 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ee1b7-c51f-49b8-b9be-78f3127d5cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:01.270227 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ee1b7-c51f-49b8-b9be-78f3127d5cb5 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:01.271996 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ee1b7-c51f-49b8-b9be-78f3127d5cb5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:01.272908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ee1b7-c51f-49b8-b9be-78f3127d5cb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:01.275932 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ee1b7-c51f-49b8-b9be-78f3127d5cb5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:01.281699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:01.293868 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 408/820] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Jun 30 18:16:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:01.294935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052ae52-2434-4633-8a4b-21e4e83b8b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:01.295549 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ee1b7-c51f-49b8-b9be-78f3127d5cb5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:01.295883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052ae52-2434-4633-8a4b-21e4e83b8b79 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:01.296245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052ae52-2434-4633-8a4b-21e4e83b8b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:01.296437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052ae52-2434-4633-8a4b-21e4e83b8b79 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:01.297305 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5052ae52-2434-4633-8a4b-21e4e83b8b79 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:01.297681 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5052ae52-2434-4633-8a4b-21e4e83b8b79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:01.299141 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5052ae52-2434-4633-8a4b-21e4e83b8b79 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:01.335399 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 413/821] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Jun 30 18:16:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:01.337458 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052ae52-2434-4633-8a4b-21e4e83b8b79 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:01.464740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ee1b7-c51f-49b8-b9be-78f3127d5cb5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:01.494846 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052ae52-2434-4633-8a4b-21e4e83b8b79 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:01.518674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1c350-3b71-4bc9-b30f-661f145401a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:01.519454 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1c350-3b71-4bc9-b30f-661f145401a1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:01.520352 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1c350-3b71-4bc9-b30f-661f145401a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:01.520765 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1c350-3b71-4bc9-b30f-661f145401a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:01.522004 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1c350-3b71-4bc9-b30f-661f145401a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:01.522501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1c350-3b71-4bc9-b30f-661f145401a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:01.524545 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1c350-3b71-4bc9-b30f-661f145401a1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:01.538441 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946f259-07f5-4ecc-bca0-cbccf1346513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:01.539046 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946f259-07f5-4ecc-bca0-cbccf1346513 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:01.539328 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946f259-07f5-4ecc-bca0-cbccf1346513 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:01.539607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946f259-07f5-4ecc-bca0-cbccf1346513 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:01.540316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9946f259-07f5-4ecc-bca0-cbccf1346513 None admin] RBAC: Adding query filter params (name=network) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:01.540614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9946f259-07f5-4ecc-bca0-cbccf1346513 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:01.548916 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 409/822] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:16:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:01.549140 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9946f259-07f5-4ecc-bca0-cbccf1346513 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:01.549238 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1c350-3b71-4bc9-b30f-661f145401a1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:01.559411 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 414/823] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 30 18:16:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:01.559411 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946f259-07f5-4ecc-bca0-cbccf1346513 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:01.619378 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:01.649691 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:01.718410 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1c350-3b71-4bc9-b30f-661f145401a1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:01.724632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946f259-07f5-4ecc-bca0-cbccf1346513 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:01.776515 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95059a62-da1d-4bb9-950a-54cd3cecb053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:01.777110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95059a62-da1d-4bb9-950a-54cd3cecb053 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:01.777395 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95059a62-da1d-4bb9-950a-54cd3cecb053 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:01.777710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95059a62-da1d-4bb9-950a-54cd3cecb053 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:01.778205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfd1e0-7565-4674-b1b7-18fef1f21d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:01.778473 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95059a62-da1d-4bb9-950a-54cd3cecb053 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:01.779685 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfd1e0-7565-4674-b1b7-18fef1f21d44 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:01.779872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95059a62-da1d-4bb9-950a-54cd3cecb053 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:01.780340 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfd1e0-7565-4674-b1b7-18fef1f21d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:01.780730 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfd1e0-7565-4674-b1b7-18fef1f21d44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:01.796708 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcfd1e0-7565-4674-b1b7-18fef1f21d44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:01.799528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcfd1e0-7565-4674-b1b7-18fef1f21d44 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:01.821607 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 410/824] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 30 18:16:01 2022] POST /identity/v3/endpoints => generated 346 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:16:01.821607 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95059a62-da1d-4bb9-950a-54cd3cecb053 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:01.833230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-9bcfd1e0-7565-4674-b1b7-18fef1f21d44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:01.833230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:01.833230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:01.833230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:01.833230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:01.833230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:01.833230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:01.833230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:01.833230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:01.833230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:01.833230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:01.833230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:01.833230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:16:01.833230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:16:01.833230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:16:01.834927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:16:01.834927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:01.834927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:01.834927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:01.834927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:01.834927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:01.834927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:01.834927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:01.834927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:01.834927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:01.834927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:01.834927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:01.834927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:01.834927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:01.834927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:01.834927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:01.834927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:01.835981 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:16:01.835981 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:16:01.835981 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:01.835981 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:01.835981 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 415/825] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:01.835981 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfd1e0-7565-4674-b1b7-18fef1f21d44 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:01.979493 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:01.984787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:01.987206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfd1e0-7565-4674-b1b7-18fef1f21d44 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:02.006966 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95059a62-da1d-4bb9-950a-54cd3cecb053 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:02.043772 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54aaf6-11f8-4a36-a04c-66ec459fd373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:02.044470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54aaf6-11f8-4a36-a04c-66ec459fd373 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:02.044758 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54aaf6-11f8-4a36-a04c-66ec459fd373 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:02.045041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54aaf6-11f8-4a36-a04c-66ec459fd373 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:02.047079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54aaf6-11f8-4a36-a04c-66ec459fd373 None admin] RBAC: Adding query filter params (type=network) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:02.047450 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54aaf6-11f8-4a36-a04c-66ec459fd373 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:02.048802 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54aaf6-11f8-4a36-a04c-66ec459fd373 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:02.055181 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 416/826] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 30 18:16:01 2022] GET /identity/v3/services?type=network => generated 350 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:02.056041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54aaf6-11f8-4a36-a04c-66ec459fd373 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:02.057085 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71869f9-b58e-44d0-8764-9cfa7e05e09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:02.058281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71869f9-b58e-44d0-8764-9cfa7e05e09a None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:02.058672 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71869f9-b58e-44d0-8764-9cfa7e05e09a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:02.059053 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71869f9-b58e-44d0-8764-9cfa7e05e09a None admin] PATH_INFO: `/v3/projects/8c7e1ee12aa14b7ab13e07822826b25b/users/4ddcf6306e704fe9ad5ed8779178ae67/roles/1ccfbd20ff5f4d068de0fe3bf93ebe6d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:02.086594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71869f9-b58e-44d0-8764-9cfa7e05e09a None admin] RBAC: Authorizing `identity:create_grant(project_id=8c7e1ee12aa14b7ab13e07822826b25b, user_id=4ddcf6306e704fe9ad5ed8779178ae67, role_id=1ccfbd20ff5f4d068de0fe3bf93ebe6d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:02.088087 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71869f9-b58e-44d0-8764-9cfa7e05e09a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:02.106400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 411/827] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:16:01 2022] PUT /identity/v3/projects/8c7e1ee12aa14b7ab13e07822826b25b/users/4ddcf6306e704fe9ad5ed8779178ae67/roles/1ccfbd20ff5f4d068de0fe3bf93ebe6d => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:16:02.107309 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71869f9-b58e-44d0-8764-9cfa7e05e09a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:02.198475 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54aaf6-11f8-4a36-a04c-66ec459fd373 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:02.243260 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e2d03-a6ed-4548-9a87-59254f633ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:02.244004 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e2d03-a6ed-4548-9a87-59254f633ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:02.244302 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e2d03-a6ed-4548-9a87-59254f633ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:02.244585 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e2d03-a6ed-4548-9a87-59254f633ea0 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:02.245344 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e2d03-a6ed-4548-9a87-59254f633ea0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:02.245683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e2d03-a6ed-4548-9a87-59254f633ea0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:02.247265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e2d03-a6ed-4548-9a87-59254f633ea0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:02.261415 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 417/828] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:02.266966 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e2d03-a6ed-4548-9a87-59254f633ea0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:02.321914 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:02.365285 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71869f9-b58e-44d0-8764-9cfa7e05e09a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:02.423726 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f5d48-3466-422c-a73b-c0a9003ee356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:02.424670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f5d48-3466-422c-a73b-c0a9003ee356 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:02.425116 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f5d48-3466-422c-a73b-c0a9003ee356 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:02.425560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f5d48-3466-422c-a73b-c0a9003ee356 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:02.427760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6f5d48-3466-422c-a73b-c0a9003ee356 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:02.429430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6f5d48-3466-422c-a73b-c0a9003ee356 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:02.453992 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e2d03-a6ed-4548-9a87-59254f633ea0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:02.457556 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 412/829] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 30 18:16:02 2022] POST /identity/v3/endpoints => generated 327 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:16:02.558301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b7999-e3c2-498e-a953-abd024abd349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:02.559719 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b7999-e3c2-498e-a953-abd024abd349 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:02.560208 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b7999-e3c2-498e-a953-abd024abd349 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:02.560708 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b7999-e3c2-498e-a953-abd024abd349 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:02.583480 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b7999-e3c2-498e-a953-abd024abd349 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:02.586346 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b7999-e3c2-498e-a953-abd024abd349 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:02.618417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-031b7999-e3c2-498e-a953-abd024abd349 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:16:02.618417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:02.618417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:02.618417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:02.618417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:02.618417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:02.618417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:02.618417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:02.618417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:02.618417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:02.618417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:02.618417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:02.618417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:16:02.618417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:16:02.618417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:16:02.619699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:16:02.619699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:02.619699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:02.619699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:02.619699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:02.619699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:02.619699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:02.619699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:02.619699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:02.619699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:02.619699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:02.619699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:02.619699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:02.619699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:02.619699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:02.619699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:02.619699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:02.620940 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:16:02.620940 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:16:02.620940 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:16:02.620940 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:16:02.620940 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:16:02.620940 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:16:02.620940 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:16:02.620940 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:02.621479 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 418/830] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:16:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:16:02.631534 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f5d48-3466-422c-a73b-c0a9003ee356 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:02.636727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:02.794834 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f5d48-3466-422c-a73b-c0a9003ee356 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:02.845947 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c24ba4-50bb-4c3c-a7ae-e66d1960803f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:02.846651 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c24ba4-50bb-4c3c-a7ae-e66d1960803f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:02.846937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c24ba4-50bb-4c3c-a7ae-e66d1960803f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:02.847217 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c24ba4-50bb-4c3c-a7ae-e66d1960803f None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:02.848022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c24ba4-50bb-4c3c-a7ae-e66d1960803f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:02.848339 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c24ba4-50bb-4c3c-a7ae-e66d1960803f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:02.850057 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c24ba4-50bb-4c3c-a7ae-e66d1960803f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:02.868934 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583451a2-504d-43b6-908b-385bb42955d6 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:02.869218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583451a2-504d-43b6-908b-385bb42955d6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:02.869498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583451a2-504d-43b6-908b-385bb42955d6 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:02.869783 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 413/831] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:16:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:02.875356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 419/832] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:16:02.875856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c24ba4-50bb-4c3c-a7ae-e66d1960803f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:02.887543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38127d20-046d-4aba-a5d9-1e781d218d87 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:02.887835 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38127d20-046d-4aba-a5d9-1e781d218d87 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:02.888113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38127d20-046d-4aba-a5d9-1e781d218d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:02.913734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-38127d20-046d-4aba-a5d9-1e781d218d87 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:02.915276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-38127d20-046d-4aba-a5d9-1e781d218d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:02.960484 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:03.011431 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c24ba4-50bb-4c3c-a7ae-e66d1960803f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:03.017926 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 420/833] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:16:03.051722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23537e-95b4-41db-b55f-3bc17440d9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:03.052874 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23537e-95b4-41db-b55f-3bc17440d9c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:03.053231 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23537e-95b4-41db-b55f-3bc17440d9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:03.053556 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23537e-95b4-41db-b55f-3bc17440d9c3 None admin] PATH_INFO: `/v3/projects/8c7e1ee12aa14b7ab13e07822826b25b/users/4bd521b9e93f4b3b8af64d46f36c019e/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:03.055935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38127d20-046d-4aba-a5d9-1e781d218d87 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:03.059093 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23537e-95b4-41db-b55f-3bc17440d9c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c7e1ee12aa14b7ab13e07822826b25b, user_id=4bd521b9e93f4b3b8af64d46f36c019e, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:03.060641 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23537e-95b4-41db-b55f-3bc17440d9c3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:03.078784 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:03.081750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 414/834] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:16:02 2022] PUT /identity/v3/projects/8c7e1ee12aa14b7ab13e07822826b25b/users/4bd521b9e93f4b3b8af64d46f36c019e/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:16:03.206926 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38127d20-046d-4aba-a5d9-1e781d218d87 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:03.254965 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2ba56-1e03-4cbf-99d4-8f5649c47f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:03.256935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2ba56-1e03-4cbf-99d4-8f5649c47f13 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:03.257682 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2ba56-1e03-4cbf-99d4-8f5649c47f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:03.258198 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2ba56-1e03-4cbf-99d4-8f5649c47f13 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:03.259465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e2ba56-1e03-4cbf-99d4-8f5649c47f13 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:03.261155 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e2ba56-1e03-4cbf-99d4-8f5649c47f13 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:03.283543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-25e2ba56-1e03-4cbf-99d4-8f5649c47f13 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 18:16:03.283543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:03.283543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:16:03.283543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:16:03.283543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:16:03.283543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:16:03.283543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:16:03.283543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:03.283543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 18:16:03.283543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:03.283543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:16:03.283543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:03.283543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:03.283543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:03.283543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:03.283543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:03.284786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:03.284786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:03.284786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:03.284786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:03.284786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:03.284786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:03.284786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:03.284786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:16:03.284786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:16:03.284786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:16:03.284786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:16:03.284786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:03.284786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:03.284786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:03.284786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:03.284786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:03.285845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:03.285845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:03.285845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:03.285845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:03.285845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:03.285845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:03.285845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:03.285845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:03.285845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:03.285845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:03.285845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:03.285845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:16:03.285845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:03.285845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 18:16:03.285845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:03.285845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 421/835] 10.222.0.33 () {56 vars in 1132 bytes} [Thu Jun 30 18:16:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:16:03.290942 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23537e-95b4-41db-b55f-3bc17440d9c3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:03.407667 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23537e-95b4-41db-b55f-3bc17440d9c3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:03.433940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:03.444849 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b31c3-3a6c-441c-a506-7345d223ba2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:03.446168 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b31c3-3a6c-441c-a506-7345d223ba2c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:03.446748 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b31c3-3a6c-441c-a506-7345d223ba2c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:03.447025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b31c3-3a6c-441c-a506-7345d223ba2c None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:03.447814 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b31c3-3a6c-441c-a506-7345d223ba2c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:03.448129 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b31c3-3a6c-441c-a506-7345d223ba2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:03.449538 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b31c3-3a6c-441c-a506-7345d223ba2c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:03.455108 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 415/836] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 30 18:16:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:03.460982 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2ba56-1e03-4cbf-99d4-8f5649c47f13 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:03.540752 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:03.569363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2ba56-1e03-4cbf-99d4-8f5649c47f13 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:03.606346 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38710bc3-3373-41d4-8234-37de61853694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:03.607006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38710bc3-3373-41d4-8234-37de61853694 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:03.607294 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38710bc3-3373-41d4-8234-37de61853694 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:03.607576 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38710bc3-3373-41d4-8234-37de61853694 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:03.608275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38710bc3-3373-41d4-8234-37de61853694 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:03.608631 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38710bc3-3373-41d4-8234-37de61853694 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:03.610129 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38710bc3-3373-41d4-8234-37de61853694 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:03.616568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 422/837] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 30 18:16:03 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:04.282771 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc170bb-341a-4fe2-9043-71df0de9e34e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:04.283524 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc170bb-341a-4fe2-9043-71df0de9e34e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:04.284042 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc170bb-341a-4fe2-9043-71df0de9e34e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:04.285985 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 416/838] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:04.291961 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3355e-4964-48c5-834e-9382ba83c8ea None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:04.292512 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3355e-4964-48c5-834e-9382ba83c8ea None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:04.292512 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3355e-4964-48c5-834e-9382ba83c8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:04.310938 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2ee3355e-4964-48c5-834e-9382ba83c8ea None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:04.312654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2ee3355e-4964-48c5-834e-9382ba83c8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:04.390040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 423/839] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:16:04.412997 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc170bb-341a-4fe2-9043-71df0de9e34e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:04.533802 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc170bb-341a-4fe2-9043-71df0de9e34e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:04.573991 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b5ba08-41ee-4f44-ae1d-b1fa9a85156a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:04.575151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5ba08-41ee-4f44-ae1d-b1fa9a85156a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:04.575622 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5ba08-41ee-4f44-ae1d-b1fa9a85156a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:04.576073 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5ba08-41ee-4f44-ae1d-b1fa9a85156a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:04.576941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b5ba08-41ee-4f44-ae1d-b1fa9a85156a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:04.578603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b5ba08-41ee-4f44-ae1d-b1fa9a85156a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:04.591638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a0b5ba08-41ee-4f44-ae1d-b1fa9a85156a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 18:16:04.591638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:04.591638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:16:04.591638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:16:04.591638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:16:04.591638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:16:04.591638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:16:04.591638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:04.591638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 18:16:04.591638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:04.591638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:16:04.591638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:04.591638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:04.591638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:04.591638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:04.591638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:04.591638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:04.592941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:04.592941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:04.592941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:04.592941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:04.592941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:04.592941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:04.592941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:16:04.592941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:16:04.592941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:16:04.592941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:16:04.592941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:04.592941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:04.592941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:04.592941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:04.592941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:04.592941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:04.593987 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:04.593987 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:04.593987 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:04.593987 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:04.593987 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:04.593987 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:04.593987 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:04.593987 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:04.593987 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:04.593987 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:04.593987 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:16:04.593987 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:04.593987 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 18:16:04.593987 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:04.594977 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace989b-ac44-49ba-8579-071a59f70988 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:04.595273 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace989b-ac44-49ba-8579-071a59f70988 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:04.595550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace989b-ac44-49ba-8579-071a59f70988 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:04.596367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 417/840] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 30 18:16:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:04.601802 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 424/841] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:04.602789 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b5ba08-41ee-4f44-ae1d-b1fa9a85156a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:04.606803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a07fe-1352-4d60-bea8-17043a898cc4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:04.607101 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a07fe-1352-4d60-bea8-17043a898cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:04.607380 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a07fe-1352-4d60-bea8-17043a898cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:04.624995 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-420a07fe-1352-4d60-bea8-17043a898cc4 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:04.626234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-420a07fe-1352-4d60-bea8-17043a898cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:04.690285 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:04.702928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:04.713708 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 425/842] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:16:04.730717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b5ba08-41ee-4f44-ae1d-b1fa9a85156a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:04.740406 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a07fe-1352-4d60-bea8-17043a898cc4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:04.770125 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941eb3e-0fe3-4186-89f8-61768d52cde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:04.771337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941eb3e-0fe3-4186-89f8-61768d52cde7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:04.771885 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941eb3e-0fe3-4186-89f8-61768d52cde7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:04.772360 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941eb3e-0fe3-4186-89f8-61768d52cde7 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:04.773491 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941eb3e-0fe3-4186-89f8-61768d52cde7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:04.774028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941eb3e-0fe3-4186-89f8-61768d52cde7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:04.775600 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941eb3e-0fe3-4186-89f8-61768d52cde7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:04.781214 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 418/843] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 30 18:16:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:04.787640 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941eb3e-0fe3-4186-89f8-61768d52cde7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:04.856335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a07fe-1352-4d60-bea8-17043a898cc4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:04.893482 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c0738-9179-4457-a095-26cf65aeeab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:04.894338 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c0738-9179-4457-a095-26cf65aeeab8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:04.894627 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c0738-9179-4457-a095-26cf65aeeab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:04.894907 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c0738-9179-4457-a095-26cf65aeeab8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:04.901108 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941eb3e-0fe3-4186-89f8-61768d52cde7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:04.906510 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c0738-9179-4457-a095-26cf65aeeab8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:16:04.914826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-907c0738-9179-4457-a095-26cf65aeeab8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:16:04.914826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:04.914826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:04.914826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:04.914826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:04.914826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:04.914826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:04.914826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:04.914826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:04.914826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:04.914826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:04.914826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:04.914826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:16:04.914826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:16:04.914826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:16:04.914826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:16:04.915760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:16:04.915760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:16:04.915760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:16:04.915760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:16:04.915760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:04.915760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:04.915760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:04.915760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:04.915760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:04.915760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:04.915760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:04.915760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:04.915760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:04.915760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:04.915760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:04.915760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:04.916593 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:04.916593 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:04.916593 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:04.916593 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:04.916593 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:16:04.916593 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:16:04.916593 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:16:04.916593 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:16:04.916593 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:16:04.916593 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:16:04.916593 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:16:04.916593 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:04.921259 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 426/844] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 30 18:16:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:16:04.922447 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:04.922562 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c0738-9179-4457-a095-26cf65aeeab8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:04.944115 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730e46a-b262-4091-812f-0a051a1859b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:04.945439 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730e46a-b262-4091-812f-0a051a1859b5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:04.945955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730e46a-b262-4091-812f-0a051a1859b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:04.946513 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730e46a-b262-4091-812f-0a051a1859b5 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:04.947413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730e46a-b262-4091-812f-0a051a1859b5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:04.947982 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730e46a-b262-4091-812f-0a051a1859b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:04.949887 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730e46a-b262-4091-812f-0a051a1859b5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:04.955809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 419/845] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 30 18:16:04 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:05.000700 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:05.054915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c0738-9179-4457-a095-26cf65aeeab8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:05.095141 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15aef72-6e91-4e88-b0e1-8331f0523660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:05.096212 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15aef72-6e91-4e88-b0e1-8331f0523660 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:05.096709 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15aef72-6e91-4e88-b0e1-8331f0523660 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:05.097223 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15aef72-6e91-4e88-b0e1-8331f0523660 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:05.098251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15aef72-6e91-4e88-b0e1-8331f0523660 None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:05.098785 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15aef72-6e91-4e88-b0e1-8331f0523660 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:05.101083 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15aef72-6e91-4e88-b0e1-8331f0523660 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:05.111265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 427/846] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 30 18:16:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:05.121638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730e46a-b262-4091-812f-0a051a1859b5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:05.289911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730e46a-b262-4091-812f-0a051a1859b5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:05.331642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1841077-12db-4cda-800b-fb7e23459163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:05.332528 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1841077-12db-4cda-800b-fb7e23459163 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:05.332912 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1841077-12db-4cda-800b-fb7e23459163 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:05.333294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1841077-12db-4cda-800b-fb7e23459163 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:05.346426 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1841077-12db-4cda-800b-fb7e23459163 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:05.348082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1841077-12db-4cda-800b-fb7e23459163 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:05.367117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d1841077-12db-4cda-800b-fb7e23459163 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:05.367117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:05.367117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:05.367117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:05.367117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:05.367117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:05.367117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:05.367117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:05.367117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:05.367117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:05.367117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:05.367117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:05.367117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:16:05.367117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:16:05.367117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:16:05.368512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:16:05.368512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:05.368512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:05.368512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:05.368512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:05.368512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:05.368512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:05.368512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:05.368512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:05.368512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:05.368512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:05.368512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:05.368512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:05.368512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:05.368512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:05.368512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:05.368512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:05.369652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:16:05.369652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:16:05.369652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:05.369652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:05.371095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 420/847] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:05.376694 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15aef72-6e91-4e88-b0e1-8331f0523660 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:05.479787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858a275-315d-4bcd-ae70-3118b97deeb4 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:05.480372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858a275-315d-4bcd-ae70-3118b97deeb4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:05.480372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858a275-315d-4bcd-ae70-3118b97deeb4 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:05.482027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 421/848] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:05.487347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15487b6c-ef91-4efe-b460-8cd60cdc0fb0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:05.487642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15487b6c-ef91-4efe-b460-8cd60cdc0fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:05.487919 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15487b6c-ef91-4efe-b460-8cd60cdc0fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:05.495343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15aef72-6e91-4e88-b0e1-8331f0523660 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:05.507361 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-15487b6c-ef91-4efe-b460-8cd60cdc0fb0 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:05.508766 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-15487b6c-ef91-4efe-b460-8cd60cdc0fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:05.518369 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:05.538186 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:05.539361 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f1b84-572c-412e-9f91-3ac44c680100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:05.540025 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f1b84-572c-412e-9f91-3ac44c680100 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:05.540272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f1b84-572c-412e-9f91-3ac44c680100 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:05.540572 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f1b84-572c-412e-9f91-3ac44c680100 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:05.541399 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f1b84-572c-412e-9f91-3ac44c680100 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:05.541805 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f1b84-572c-412e-9f91-3ac44c680100 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:05.543865 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f1b84-572c-412e-9f91-3ac44c680100 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:05.558098 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 428/849] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:05.567322 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f1b84-572c-412e-9f91-3ac44c680100 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:05.593512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 422/850] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:05 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:16:05.618913 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15487b6c-ef91-4efe-b460-8cd60cdc0fb0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:05.702863 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f1b84-572c-412e-9f91-3ac44c680100 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:05.750083 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15487b6c-ef91-4efe-b460-8cd60cdc0fb0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:05.750325 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7da01a-31ae-4137-ac42-5e47463f8a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:05.751086 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7da01a-31ae-4137-ac42-5e47463f8a5d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:05.751491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7da01a-31ae-4137-ac42-5e47463f8a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:05.751774 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7da01a-31ae-4137-ac42-5e47463f8a5d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:05.767789 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7da01a-31ae-4137-ac42-5e47463f8a5d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:05.769679 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7da01a-31ae-4137-ac42-5e47463f8a5d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:05.795906 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaddc5ef-15be-4192-b7c6-d8e49ec84693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:05.796961 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddc5ef-15be-4192-b7c6-d8e49ec84693 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:05.797395 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddc5ef-15be-4192-b7c6-d8e49ec84693 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:05.797744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-dd7da01a-31ae-4137-ac42-5e47463f8a5d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 18:16:05.797744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:05.797744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:05.797744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:05.797744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:05.797744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:05.797744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:05.797744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:05.797744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:05.797744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:05.797744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:05.797744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:05.797744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:16:05.797744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:16:05.797744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:16:05.797744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:16:05.798819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:05.798819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:05.798819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:16:05.798819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:16:05.798819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:16:05.798819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:16:05.798819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:05.798819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:05.798819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:05.798819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:05.798819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:05.798819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:05.798819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:05.798819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:05.798819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:05.798819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:05.798819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:05.798819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:05.799854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:05.799854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:05.799854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:16:05.799854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:16:05.799854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:16:05.799854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:16:05.799854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:16:05.799854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:16:05.799854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 18:16:05.799854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:05.800687 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddc5ef-15be-4192-b7c6-d8e49ec84693 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:05.801715 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 429/851] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 30 18:16:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:05.805681 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7da01a-31ae-4137-ac42-5e47463f8a5d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:05.818853 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaddc5ef-15be-4192-b7c6-d8e49ec84693 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:16:05.826930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-eaddc5ef-15be-4192-b7c6-d8e49ec84693 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:16:05.826930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:05.826930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:05.826930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:05.826930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:05.826930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:05.826930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:05.826930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:05.826930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:05.826930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:05.826930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:05.826930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:05.826930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:16:05.826930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:16:05.826930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:16:05.826930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:16:05.828110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:16:05.828110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:16:05.828110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:16:05.828110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:16:05.828110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:05.828110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:05.828110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:05.828110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:05.828110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:05.828110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:05.828110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:05.828110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:05.828110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:05.828110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:05.828110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:05.828110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:05.829270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:05.829270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:05.829270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:05.829270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:05.829270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:16:05.829270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:16:05.829270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:16:05.829270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:16:05.829270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:16:05.829270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:16:05.829270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:16:05.829270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:05.832782 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 423/852] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:16:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:05.834184 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3acd87-0f3f-407f-a8ec-afbf951ab9d7 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:05.834474 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3acd87-0f3f-407f-a8ec-afbf951ab9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:05.834855 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3acd87-0f3f-407f-a8ec-afbf951ab9d7 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:05.836299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 424/853] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:05.836871 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3acd87-0f3f-407f-a8ec-afbf951ab9d7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:05.859627 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:05.863611 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:05.928932 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7da01a-31ae-4137-ac42-5e47463f8a5d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:05.948000 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3acd87-0f3f-407f-a8ec-afbf951ab9d7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:05.968932 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4c6c9-df2c-4dcc-984f-6a2ed7d916f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:05.969790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4c6c9-df2c-4dcc-984f-6a2ed7d916f1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:05.970081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4c6c9-df2c-4dcc-984f-6a2ed7d916f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:05.970361 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4c6c9-df2c-4dcc-984f-6a2ed7d916f1 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:05.971190 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e4c6c9-df2c-4dcc-984f-6a2ed7d916f1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:05.971509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e4c6c9-df2c-4dcc-984f-6a2ed7d916f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:05.972951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e4c6c9-df2c-4dcc-984f-6a2ed7d916f1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:05.984357 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1908c6-d84d-4129-8c10-8c07ba4a0e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:05.985019 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1908c6-d84d-4129-8c10-8c07ba4a0e37 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:05.985305 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1908c6-d84d-4129-8c10-8c07ba4a0e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:05.985603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1908c6-d84d-4129-8c10-8c07ba4a0e37 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:05.986350 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1908c6-d84d-4129-8c10-8c07ba4a0e37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:05.986661 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1908c6-d84d-4129-8c10-8c07ba4a0e37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:05.988032 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1908c6-d84d-4129-8c10-8c07ba4a0e37 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:05.996820 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 430/854] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Jun 30 18:16:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:05.998403 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939a1c4-5846-4f50-ae9a-e7c5d204f508 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:05.998514 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 425/855] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:16:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:05.998997 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939a1c4-5846-4f50-ae9a-e7c5d204f508 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:05.999252 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1908c6-d84d-4129-8c10-8c07ba4a0e37 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:05.999735 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2939a1c4-5846-4f50-ae9a-e7c5d204f508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:06.023578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2939a1c4-5846-4f50-ae9a-e7c5d204f508 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:06.024893 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2939a1c4-5846-4f50-ae9a-e7c5d204f508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:06.099080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:06.101872 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 431/856] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:05 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:06.103127 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2939a1c4-5846-4f50-ae9a-e7c5d204f508 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:06.122060 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1908c6-d84d-4129-8c10-8c07ba4a0e37 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:06.140539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:06.157705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4adb0bc-2d79-4a7b-8197-46cd9f8be971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:06.158568 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4adb0bc-2d79-4a7b-8197-46cd9f8be971 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:06.158865 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4adb0bc-2d79-4a7b-8197-46cd9f8be971 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:06.159138 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4adb0bc-2d79-4a7b-8197-46cd9f8be971 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:06.223562 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2939a1c4-5846-4f50-ae9a-e7c5d204f508 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:06.237046 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4adb0bc-2d79-4a7b-8197-46cd9f8be971 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:06.238672 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4adb0bc-2d79-4a7b-8197-46cd9f8be971 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:06.265035 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3b1b5-50a9-4e08-bbcf-0644ceb4d7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:06.265867 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3b1b5-50a9-4e08-bbcf-0644ceb4d7bf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:06.266187 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3b1b5-50a9-4e08-bbcf-0644ceb4d7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:06.266470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3b1b5-50a9-4e08-bbcf-0644ceb4d7bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:06.278131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3b1b5-50a9-4e08-bbcf-0644ceb4d7bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:06.280097 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3b1b5-50a9-4e08-bbcf-0644ceb4d7bf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:06.301399 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-44f3b1b5-50a9-4e08-bbcf-0644ceb4d7bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:06.301399 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:06.301399 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:06.301399 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:06.301399 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:06.301399 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:06.301399 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:06.301399 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:06.301399 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:06.301399 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:06.301399 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:06.301399 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:06.301399 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:16:06.301399 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:16:06.301399 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:16:06.304110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:16:06.304110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:06.304110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:06.304110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:06.304110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:06.304110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:06.304110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:06.304110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:06.304110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:06.304110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:06.304110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:06.304110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:06.304110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:06.304110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:06.304110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:06.304110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:06.304110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:06.305062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:16:06.305062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:16:06.305062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:06.305062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:06.305062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 432/857] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:06.305062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3b1b5-50a9-4e08-bbcf-0644ceb4d7bf None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:06.350964 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:06.389243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 426/858] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:16:05 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 18:16:06.389243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4adb0bc-2d79-4a7b-8197-46cd9f8be971 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:06.402489 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:06.424167 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3b1b5-50a9-4e08-bbcf-0644ceb4d7bf None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:06.484654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0c9f9-00d1-4b29-a517-0bf9a16dd50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:06.485840 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0c9f9-00d1-4b29-a517-0bf9a16dd50b None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:06.486351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0c9f9-00d1-4b29-a517-0bf9a16dd50b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:06.486863 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0c9f9-00d1-4b29-a517-0bf9a16dd50b None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:06.488263 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f0c9f9-00d1-4b29-a517-0bf9a16dd50b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:06.491361 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f0c9f9-00d1-4b29-a517-0bf9a16dd50b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:06.517646 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 433/859] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 30 18:16:06 2022] POST /identity/v3/services => generated 269 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:16:06.517646 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0c9f9-00d1-4b29-a517-0bf9a16dd50b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:06.571175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4adb0bc-2d79-4a7b-8197-46cd9f8be971 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:06.625208 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884f08b-351c-4c58-8b17-430df95f9c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:06.626281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884f08b-351c-4c58-8b17-430df95f9c03 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:06.626749 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884f08b-351c-4c58-8b17-430df95f9c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:06.627222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884f08b-351c-4c58-8b17-430df95f9c03 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:06.628284 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884f08b-351c-4c58-8b17-430df95f9c03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:06.628787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884f08b-351c-4c58-8b17-430df95f9c03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:06.631248 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884f08b-351c-4c58-8b17-430df95f9c03 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:06.653327 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 427/860] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:06.657284 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884f08b-351c-4c58-8b17-430df95f9c03 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:06.716462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0c9f9-00d1-4b29-a517-0bf9a16dd50b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:06.746132 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:06.781244 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615477b0-85a1-470b-90a0-c5a340388e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:06.782283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615477b0-85a1-470b-90a0-c5a340388e6c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:06.783208 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615477b0-85a1-470b-90a0-c5a340388e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:06.783208 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615477b0-85a1-470b-90a0-c5a340388e6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:06.796200 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:06.800298 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615477b0-85a1-470b-90a0-c5a340388e6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:06.803633 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615477b0-85a1-470b-90a0-c5a340388e6c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:06.835916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-615477b0-85a1-470b-90a0-c5a340388e6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:06.835916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:06.835916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:06.835916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:06.835916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:06.835916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:06.835916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:06.835916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:06.835916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:06.835916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:06.835916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:06.835916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:06.835916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:16:06.835916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:16:06.835916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:16:06.837111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:16:06.837111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:06.837111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:06.837111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:06.837111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:06.837111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:06.837111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:06.837111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:06.837111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:06.837111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:06.837111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:06.837111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:06.837111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:06.837111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:06.837111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:06.837111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:06.837111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:06.838503 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:16:06.838503 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:16:06.838503 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:06.838503 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:06.839645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 434/861] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:06.839974 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884f08b-351c-4c58-8b17-430df95f9c03 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:06.851035 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615477b0-85a1-470b-90a0-c5a340388e6c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:06.896977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d1217-cc6a-498a-a31f-5b565d3aa6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:06.898017 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d1217-cc6a-498a-a31f-5b565d3aa6e5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:06.898370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d1217-cc6a-498a-a31f-5b565d3aa6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:06.898717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d1217-cc6a-498a-a31f-5b565d3aa6e5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:06.908133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303d1217-cc6a-498a-a31f-5b565d3aa6e5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:06.909705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303d1217-cc6a-498a-a31f-5b565d3aa6e5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:06.926236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-303d1217-cc6a-498a-a31f-5b565d3aa6e5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 18:16:06.926236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:06.926236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:06.926236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:06.926236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:06.926236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:06.926236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:06.926236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:06.926236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:06.926236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:06.926236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:06.926236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:06.926236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:16:06.926236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:16:06.926236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:16:06.926236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:16:06.927274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:06.927274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:06.927274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:16:06.927274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:16:06.927274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:16:06.927274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:16:06.927274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:06.927274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:06.927274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:06.927274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:06.927274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:06.927274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:06.927274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:06.927274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:06.927274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:06.927274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:06.927274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:06.927274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:06.928794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:06.928794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:06.928794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:16:06.928794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:16:06.928794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:16:06.928794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:16:06.928794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:16:06.928794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:16:06.928794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 18:16:06.928794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:06.929418 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 428/862] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:16:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:06.934912 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d1217-cc6a-498a-a31f-5b565d3aa6e5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:06.993774 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615477b0-85a1-470b-90a0-c5a340388e6c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:07.041298 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591fad4-e9a9-46c7-abdb-e6c5477af301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:07.042342 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591fad4-e9a9-46c7-abdb-e6c5477af301 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:07.042458 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591fad4-e9a9-46c7-abdb-e6c5477af301 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:07.042648 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591fad4-e9a9-46c7-abdb-e6c5477af301 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:07.043378 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591fad4-e9a9-46c7-abdb-e6c5477af301 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:07.043687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591fad4-e9a9-46c7-abdb-e6c5477af301 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:07.045771 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591fad4-e9a9-46c7-abdb-e6c5477af301 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:07.064708 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 435/863] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:07.065722 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591fad4-e9a9-46c7-abdb-e6c5477af301 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:07.074702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:07.080029 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d1217-cc6a-498a-a31f-5b565d3aa6e5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:07.127870 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:07.128928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1ddd8-1b93-41e5-8d1f-31dd298a49f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:07.130391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1ddd8-1b93-41e5-8d1f-31dd298a49f5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:07.130643 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1ddd8-1b93-41e5-8d1f-31dd298a49f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:07.130969 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1ddd8-1b93-41e5-8d1f-31dd298a49f5 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:07.132078 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1ddd8-1b93-41e5-8d1f-31dd298a49f5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:07.132206 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1ddd8-1b93-41e5-8d1f-31dd298a49f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:07.135011 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1ddd8-1b93-41e5-8d1f-31dd298a49f5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:07.158427 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 429/864] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Jun 30 18:16:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:07.159652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1ddd8-1b93-41e5-8d1f-31dd298a49f5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:07.205402 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591fad4-e9a9-46c7-abdb-e6c5477af301 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:07.252406 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7680e68-f833-46f8-9b6c-289d391b4300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:07.253225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7680e68-f833-46f8-9b6c-289d391b4300 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:07.253505 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7680e68-f833-46f8-9b6c-289d391b4300 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:07.253807 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7680e68-f833-46f8-9b6c-289d391b4300 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:07.261261 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7680e68-f833-46f8-9b6c-289d391b4300 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:07.262683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7680e68-f833-46f8-9b6c-289d391b4300 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:07.287570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b7680e68-f833-46f8-9b6c-289d391b4300 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:16:07.287570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:07.287570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:07.287570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:07.287570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:07.287570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:07.287570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:07.287570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:07.287570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:07.287570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:07.287570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:07.287570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:07.287570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:16:07.287570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:16:07.287570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:16:07.289281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:16:07.289281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:07.289281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:07.289281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:07.289281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:07.289281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:07.289281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:07.289281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:07.289281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:07.289281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:07.289281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:07.289281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:07.289281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:07.289281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:07.289281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:07.289281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:07.289281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:07.293292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:16:07.293292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:16:07.293292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:16:07.293292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:16:07.293292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:16:07.293292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:16:07.293292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:16:07.293292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:07.294217 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1ddd8-1b93-41e5-8d1f-31dd298a49f5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:07.297223 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 436/865] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:16:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:07.300004 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc62145-3789-4d91-8a9e-be40f4949f95 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:07.300528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc62145-3789-4d91-8a9e-be40f4949f95 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:07.301449 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc62145-3789-4d91-8a9e-be40f4949f95 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:07.303978 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 437/866] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:07.304887 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc62145-3789-4d91-8a9e-be40f4949f95 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:07.327305 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0703d-59ae-4699-bbd4-42e67f35cd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:07.327952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0703d-59ae-4699-bbd4-42e67f35cd83 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:07.328234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0703d-59ae-4699-bbd4-42e67f35cd83 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:07.328515 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0703d-59ae-4699-bbd4-42e67f35cd83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:07.356201 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:07.405533 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f0703d-59ae-4699-bbd4-42e67f35cd83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:07.407727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f0703d-59ae-4699-bbd4-42e67f35cd83 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:07.426768 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc62145-3789-4d91-8a9e-be40f4949f95 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:07.441729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:07.468210 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc4a9e-723b-4d12-b0b1-24b29f012c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:07.469041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc4a9e-723b-4d12-b0b1-24b29f012c5c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:07.469557 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc4a9e-723b-4d12-b0b1-24b29f012c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:07.469844 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc4a9e-723b-4d12-b0b1-24b29f012c5c None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:07.470768 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc4a9e-723b-4d12-b0b1-24b29f012c5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:07.471180 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc4a9e-723b-4d12-b0b1-24b29f012c5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:07.472746 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc4a9e-723b-4d12-b0b1-24b29f012c5c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:07.505355 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 438/867] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:16:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:07.505355 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45c62e-278f-4730-87a9-4a85045b0dd2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:07.505355 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45c62e-278f-4730-87a9-4a85045b0dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:07.505355 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45c62e-278f-4730-87a9-4a85045b0dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:07.526108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-eb45c62e-278f-4730-87a9-4a85045b0dd2 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:07.527345 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-eb45c62e-278f-4730-87a9-4a85045b0dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:07.628537 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 439/868] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:07 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:07.629449 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45c62e-278f-4730-87a9-4a85045b0dd2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:07.665382 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 430/869] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:16:07 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 18:16:07.666862 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0703d-59ae-4699-bbd4-42e67f35cd83 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:07.742416 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:07.759610 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45c62e-278f-4730-87a9-4a85045b0dd2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:07.761397 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:07.805114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8536e64-8c97-4f8d-806c-1bfb84286d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:07.805822 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8536e64-8c97-4f8d-806c-1bfb84286d27 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:07.806181 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8536e64-8c97-4f8d-806c-1bfb84286d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:07.806402 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8536e64-8c97-4f8d-806c-1bfb84286d27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:07.807245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8536e64-8c97-4f8d-806c-1bfb84286d27 None admin] RBAC: Adding query filter params (role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d, scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b, user.id=4ddcf6306e704fe9ad5ed8779178ae67) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:07.807756 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8536e64-8c97-4f8d-806c-1bfb84286d27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:07.809099 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8536e64-8c97-4f8d-806c-1bfb84286d27 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:07.816339 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 440/870] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:16:07 2022] GET /identity/v3/role_assignments?user.id=4ddcf6306e704fe9ad5ed8779178ae67&scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b&role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:07.817192 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8536e64-8c97-4f8d-806c-1bfb84286d27 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:07.830871 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0703d-59ae-4699-bbd4-42e67f35cd83 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:07.888000 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650b056-1bce-4a45-849f-ec4139f71226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:07.889145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650b056-1bce-4a45-849f-ec4139f71226 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:07.889758 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650b056-1bce-4a45-849f-ec4139f71226 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:07.890884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650b056-1bce-4a45-849f-ec4139f71226 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:07.892075 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f650b056-1bce-4a45-849f-ec4139f71226 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:07.893986 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f650b056-1bce-4a45-849f-ec4139f71226 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:07.933649 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 431/871] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 30 18:16:07 2022] POST /identity/v3/services => generated 251 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:16:07.972448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8536e64-8c97-4f8d-806c-1bfb84286d27 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:08.028782 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aebda7-8d41-4d11-acdc-faabf693782c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:08.029527 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aebda7-8d41-4d11-acdc-faabf693782c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:08.030000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aebda7-8d41-4d11-acdc-faabf693782c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:08.030347 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aebda7-8d41-4d11-acdc-faabf693782c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:08.043383 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aebda7-8d41-4d11-acdc-faabf693782c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:08.045003 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aebda7-8d41-4d11-acdc-faabf693782c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:08.066267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b9aebda7-8d41-4d11-acdc-faabf693782c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:08.066267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:08.066267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:08.066267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:08.066267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:08.066267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:08.066267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:08.066267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:08.066267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:08.066267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:08.066267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:08.066267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:08.066267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:16:08.066267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:16:08.066267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:16:08.067218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:16:08.067218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:08.067218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:08.067218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:08.067218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:08.067218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:08.067218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:08.067218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:08.067218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:08.067218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:08.067218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:08.067218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:08.067218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:08.067218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:08.067218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:08.067218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:08.067218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:08.068274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:16:08.068274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:16:08.068274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:08.068274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:08.068274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 441/872] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:08.074787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650b056-1bce-4a45-849f-ec4139f71226 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:08.206836 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650b056-1bce-4a45-849f-ec4139f71226 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:08.229805 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:08.255641 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca71b6-cf52-407f-9944-51caa8fc6e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:08.256510 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca71b6-cf52-407f-9944-51caa8fc6e08 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:08.256937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca71b6-cf52-407f-9944-51caa8fc6e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:08.257373 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca71b6-cf52-407f-9944-51caa8fc6e08 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:08.258153 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca71b6-cf52-407f-9944-51caa8fc6e08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:08.258501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca71b6-cf52-407f-9944-51caa8fc6e08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:08.260523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca71b6-cf52-407f-9944-51caa8fc6e08 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:08.275383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 432/873] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:08.281027 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aebda7-8d41-4d11-acdc-faabf693782c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:08.299547 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:08.401473 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aebda7-8d41-4d11-acdc-faabf693782c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:08.441672 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5bc07d-d64c-48f7-b1b3-25c41c1e916f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:08.442524 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5bc07d-d64c-48f7-b1b3-25c41c1e916f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:08.442818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5bc07d-d64c-48f7-b1b3-25c41c1e916f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:08.443633 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5bc07d-d64c-48f7-b1b3-25c41c1e916f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:08.451679 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5bc07d-d64c-48f7-b1b3-25c41c1e916f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:08.453227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5bc07d-d64c-48f7-b1b3-25c41c1e916f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:08.468151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3e5bc07d-d64c-48f7-b1b3-25c41c1e916f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:16:08.468151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:08.468151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:08.468151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:08.468151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:08.468151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:08.468151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:08.468151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:08.468151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:08.468151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:08.468151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:08.468151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:08.468151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:16:08.468151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:16:08.468151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:16:08.469123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:16:08.469123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:08.469123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:08.469123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:08.469123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:08.469123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:08.469123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:08.469123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:08.469123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:08.469123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:08.469123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:08.469123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:08.469123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:08.469123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:08.469123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:08.469123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:08.469123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:08.470134 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:16:08.470134 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:16:08.470134 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:16:08.470134 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:16:08.470134 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:16:08.470134 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:16:08.470134 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:16:08.470134 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:08.473485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 442/874] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:16:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:08.476694 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca71b6-cf52-407f-9944-51caa8fc6e08 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:08.636849 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca71b6-cf52-407f-9944-51caa8fc6e08 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:08.692676 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697e620-f75f-4ce0-afc0-6e3e923d65b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:08.694005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697e620-f75f-4ce0-afc0-6e3e923d65b6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:08.694385 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697e620-f75f-4ce0-afc0-6e3e923d65b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:08.694665 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697e620-f75f-4ce0-afc0-6e3e923d65b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:08.695466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e697e620-f75f-4ce0-afc0-6e3e923d65b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:08.695780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e697e620-f75f-4ce0-afc0-6e3e923d65b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:08.697238 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e697e620-f75f-4ce0-afc0-6e3e923d65b6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:08.710521 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 433/875] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:16:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:08.716469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5bc07d-d64c-48f7-b1b3-25c41c1e916f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:08.801566 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:08.805604 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3123e7-bae1-4740-87d2-084819d87b04 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:08.806027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3123e7-bae1-4740-87d2-084819d87b04 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:08.806866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3123e7-bae1-4740-87d2-084819d87b04 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:08.808245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 434/876] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:16:08.815402 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcff209-d32c-4e52-9793-3111dc122b5b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:08.816283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcff209-d32c-4e52-9793-3111dc122b5b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:08.816687 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcff209-d32c-4e52-9793-3111dc122b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:08.839383 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5bc07d-d64c-48f7-b1b3-25c41c1e916f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:08.840410 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1bcff209-d32c-4e52-9793-3111dc122b5b None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:08.842066 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1bcff209-d32c-4e52-9793-3111dc122b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:08.878680 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae281dce-adbc-44b0-ac5a-645355e7a188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:08.879593 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae281dce-adbc-44b0-ac5a-645355e7a188 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:08.880028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae281dce-adbc-44b0-ac5a-645355e7a188 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:08.880502 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae281dce-adbc-44b0-ac5a-645355e7a188 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:08.881770 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae281dce-adbc-44b0-ac5a-645355e7a188 None admin] RBAC: Adding query filter params (role.id=0ef3687af20a443281e3cfdceca3597d, scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b, user.id=4bd521b9e93f4b3b8af64d46f36c019e) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:08.882240 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae281dce-adbc-44b0-ac5a-645355e7a188 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:08.883833 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae281dce-adbc-44b0-ac5a-645355e7a188 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:08.885298 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:08.892523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 443/877] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:16:08 2022] GET /identity/v3/role_assignments?user.id=4bd521b9e93f4b3b8af64d46f36c019e&scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b&role.id=0ef3687af20a443281e3cfdceca3597d => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:08.940499 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 435/878] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:08 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:08.965328 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae281dce-adbc-44b0-ac5a-645355e7a188 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:09.095135 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae281dce-adbc-44b0-ac5a-645355e7a188 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:09.136775 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71145bc0-bc7f-43de-ae78-be4803b0e3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:09.137634 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71145bc0-bc7f-43de-ae78-be4803b0e3dd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:09.137920 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71145bc0-bc7f-43de-ae78-be4803b0e3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:09.138206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71145bc0-bc7f-43de-ae78-be4803b0e3dd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:09.138868 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71145bc0-bc7f-43de-ae78-be4803b0e3dd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:09.140272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71145bc0-bc7f-43de-ae78-be4803b0e3dd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:09.153911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-71145bc0-bc7f-43de-ae78-be4803b0e3dd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 18:16:09.153911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:09.153911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:16:09.153911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:16:09.153911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:16:09.153911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:16:09.153911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:16:09.153911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:09.153911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 18:16:09.153911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:09.153911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:16:09.153911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:09.153911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:09.153911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:09.153911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:09.153911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:09.154915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:09.154915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:09.154915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:09.154915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:09.154915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:09.154915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:09.154915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:09.154915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:16:09.154915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:16:09.154915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:16:09.154915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:16:09.154915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:09.154915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:09.154915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:09.154915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:09.154915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:09.155790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:09.155790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:09.155790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:09.155790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:09.155790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:09.155790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:09.155790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:09.155790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:09.155790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:09.155790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:09.155790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:09.155790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:16:09.155790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:09.155790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 18:16:09.155790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:09.155790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 444/879] 10.222.0.33 () {56 vars in 1132 bytes} [Thu Jun 30 18:16:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:09.161153 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcff209-d32c-4e52-9793-3111dc122b5b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:09.277098 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcff209-d32c-4e52-9793-3111dc122b5b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:09.316049 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66580849-f1bf-4c25-ab40-e243d11f363a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:09.316773 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66580849-f1bf-4c25-ab40-e243d11f363a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:09.317064 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66580849-f1bf-4c25-ab40-e243d11f363a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:09.317349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66580849-f1bf-4c25-ab40-e243d11f363a None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:09.318317 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66580849-f1bf-4c25-ab40-e243d11f363a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:09.318628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66580849-f1bf-4c25-ab40-e243d11f363a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:09.320033 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66580849-f1bf-4c25-ab40-e243d11f363a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:09.325634 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 436/880] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 30 18:16:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:09.331597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71145bc0-bc7f-43de-ae78-be4803b0e3dd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:09.335783 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:09.439901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71145bc0-bc7f-43de-ae78-be4803b0e3dd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:09.473962 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7de9cd-0164-4ab7-80c5-97c6fbbbb9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:09.474785 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7de9cd-0164-4ab7-80c5-97c6fbbbb9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:09.475014 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7de9cd-0164-4ab7-80c5-97c6fbbbb9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:09.475292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7de9cd-0164-4ab7-80c5-97c6fbbbb9a9 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:09.476005 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7de9cd-0164-4ab7-80c5-97c6fbbbb9a9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:09.476319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7de9cd-0164-4ab7-80c5-97c6fbbbb9a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:09.477729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7de9cd-0164-4ab7-80c5-97c6fbbbb9a9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:09.483124 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 445/881] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 30 18:16:09 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:09.489150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66580849-f1bf-4c25-ab40-e243d11f363a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:09.523069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:09.599835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66580849-f1bf-4c25-ab40-e243d11f363a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:09.636464 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd9f378-d0a6-4c7c-92f7-b4ed59a4658a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:09.637128 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9f378-d0a6-4c7c-92f7-b4ed59a4658a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:09.637416 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9f378-d0a6-4c7c-92f7-b4ed59a4658a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:09.637725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9f378-d0a6-4c7c-92f7-b4ed59a4658a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:09.638523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd9f378-d0a6-4c7c-92f7-b4ed59a4658a None admin] RBAC: Adding query filter params (interface=public, service_id=859bf3c671fd4772be6d012962f59b57, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:09.638840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd9f378-d0a6-4c7c-92f7-b4ed59a4658a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:09.640228 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd9f378-d0a6-4c7c-92f7-b4ed59a4658a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:09.647002 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 437/882] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 30 18:16:09 2022] GET /identity/v3/endpoints?service_id=859bf3c671fd4772be6d012962f59b57&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:09.652711 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7de9cd-0164-4ab7-80c5-97c6fbbbb9a9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:09.731868 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:09.776308 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7de9cd-0164-4ab7-80c5-97c6fbbbb9a9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:09.816671 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1540b42-b231-4886-910f-ab4d069755e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:09.817764 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1540b42-b231-4886-910f-ab4d069755e1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:09.818494 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1540b42-b231-4886-910f-ab4d069755e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:09.819542 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1540b42-b231-4886-910f-ab4d069755e1 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:09.820445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1540b42-b231-4886-910f-ab4d069755e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:09.822111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1540b42-b231-4886-910f-ab4d069755e1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:09.829105 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 446/883] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 30 18:16:09 2022] GET /identity/v3/services => generated 1268 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:16:10.221133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eba2dc-828b-4f38-b0db-5564f1efe3b8 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:10.221674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eba2dc-828b-4f38-b0db-5564f1efe3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:10.222200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eba2dc-828b-4f38-b0db-5564f1efe3b8 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:10.224152 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 438/884] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:10.230547 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87561ad4-c919-4297-8fa8-7c3f9c54942b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:10.230837 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87561ad4-c919-4297-8fa8-7c3f9c54942b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:10.231117 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87561ad4-c919-4297-8fa8-7c3f9c54942b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:10.251920 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-87561ad4-c919-4297-8fa8-7c3f9c54942b None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:10.253205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-87561ad4-c919-4297-8fa8-7c3f9c54942b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:10.349786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 447/885] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:10.367579 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eba2dc-828b-4f38-b0db-5564f1efe3b8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:10.374217 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b076e-6f74-45dd-acb2-86ce326edda4 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:10.374791 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b076e-6f74-45dd-acb2-86ce326edda4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:10.375275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b076e-6f74-45dd-acb2-86ce326edda4 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:10.377812 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 448/886] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:10.386031 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f03c9-fe0a-424f-b35d-3be2aac1e36d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:10.386967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f03c9-fe0a-424f-b35d-3be2aac1e36d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:10.387455 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f03c9-fe0a-424f-b35d-3be2aac1e36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:10.408512 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ed3f03c9-fe0a-424f-b35d-3be2aac1e36d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:10.409737 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ed3f03c9-fe0a-424f-b35d-3be2aac1e36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:10.475128 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:10.497389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 449/887] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:16:10.500974 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:10.509557 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eba2dc-828b-4f38-b0db-5564f1efe3b8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:10.518208 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f03c9-fe0a-424f-b35d-3be2aac1e36d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:10.549723 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a650438-97e6-4408-8375-dd94d1ab0f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:10.550860 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a650438-97e6-4408-8375-dd94d1ab0f7a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:10.551288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a650438-97e6-4408-8375-dd94d1ab0f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:10.551738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a650438-97e6-4408-8375-dd94d1ab0f7a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:10.563551 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a650438-97e6-4408-8375-dd94d1ab0f7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:16:10.572713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-8a650438-97e6-4408-8375-dd94d1ab0f7a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 18:16:10.572713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:10.572713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:10.572713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:10.572713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:10.572713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:10.572713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:10.572713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:10.572713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:10.572713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:10.572713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:10.572713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:10.572713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:16:10.572713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:16:10.572713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:16:10.572713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:16:10.573701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:16:10.573701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:16:10.573701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:16:10.573701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:16:10.573701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:10.573701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:10.573701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:10.573701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:10.573701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:10.573701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:10.573701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:10.573701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:10.573701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:10.573701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:10.573701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:10.573701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:10.574610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:10.574610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:10.574610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:10.574610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:10.574610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:16:10.574610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:16:10.574610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:16:10.574610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:16:10.574610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:16:10.574610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:16:10.574610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 18:16:10.574610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:10.582017 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 439/888] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Jun 30 18:16:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:10.583600 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a650438-97e6-4408-8375-dd94d1ab0f7a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:10.632398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f03c9-fe0a-424f-b35d-3be2aac1e36d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:10.671360 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21e474-d40b-4374-abe7-e8cf8ba87ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:10.672347 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21e474-d40b-4374-abe7-e8cf8ba87ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:10.672709 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21e474-d40b-4374-abe7-e8cf8ba87ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:10.673068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21e474-d40b-4374-abe7-e8cf8ba87ef2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:10.673860 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa21e474-d40b-4374-abe7-e8cf8ba87ef2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:10.675458 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa21e474-d40b-4374-abe7-e8cf8ba87ef2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:10.689296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-aa21e474-d40b-4374-abe7-e8cf8ba87ef2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 18:16:10.689296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:10.689296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:16:10.689296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:16:10.689296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:16:10.689296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:16:10.689296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:16:10.689296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:10.689296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 18:16:10.689296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:10.689296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:16:10.689296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:10.689296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:10.689296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:10.689296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:10.689296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:10.689296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:10.690389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:10.690389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:10.690389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:10.690389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:10.690389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:10.690389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:10.690389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:16:10.690389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:16:10.690389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:16:10.690389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:16:10.690389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:10.690389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:10.690389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:10.690389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:10.690389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:10.690389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:10.691231 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:10.691231 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:10.691231 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:10.691231 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:10.691231 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:10.691231 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:10.691231 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:10.691231 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:10.691231 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:10.691231 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:10.691231 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:16:10.691231 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:10.691231 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 18:16:10.691231 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:10.696532 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 450/889] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 30 18:16:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:16:10.697662 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21e474-d40b-4374-abe7-e8cf8ba87ef2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:10.704153 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a650438-97e6-4408-8375-dd94d1ab0f7a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:10.744845 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d8c21-cc64-4b68-a83d-152e544329b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:10.745977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d8c21-cc64-4b68-a83d-152e544329b7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:10.746670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d8c21-cc64-4b68-a83d-152e544329b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:10.747144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d8c21-cc64-4b68-a83d-152e544329b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:10.748102 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d8c21-cc64-4b68-a83d-152e544329b7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:10.748595 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d8c21-cc64-4b68-a83d-152e544329b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:10.750442 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d8c21-cc64-4b68-a83d-152e544329b7 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:10.760275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:10.767402 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 440/890] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 30 18:16:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:10.770272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d8c21-cc64-4b68-a83d-152e544329b7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:10.794052 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:10.814627 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21e474-d40b-4374-abe7-e8cf8ba87ef2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:10.852928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba49cf2-813e-4f78-846f-3d7f09e8e5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:10.853745 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba49cf2-813e-4f78-846f-3d7f09e8e5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:10.854124 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba49cf2-813e-4f78-846f-3d7f09e8e5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:10.854501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba49cf2-813e-4f78-846f-3d7f09e8e5c5 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:10.855299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba49cf2-813e-4f78-846f-3d7f09e8e5c5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:10.855683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba49cf2-813e-4f78-846f-3d7f09e8e5c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:10.857813 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba49cf2-813e-4f78-846f-3d7f09e8e5c5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:10.868381 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 451/891] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 30 18:16:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:10.869436 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba49cf2-813e-4f78-846f-3d7f09e8e5c5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:10.891281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d8c21-cc64-4b68-a83d-152e544329b7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:10.930304 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5032cc-1caf-4214-a031-49020ae975cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:10.931399 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5032cc-1caf-4214-a031-49020ae975cf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:10.931891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5032cc-1caf-4214-a031-49020ae975cf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:10.932341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5032cc-1caf-4214-a031-49020ae975cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:10.945104 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5032cc-1caf-4214-a031-49020ae975cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:10.947544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5032cc-1caf-4214-a031-49020ae975cf None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:10.967889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-dc5032cc-1caf-4214-a031-49020ae975cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:10.967889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:10.967889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:10.967889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:10.967889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:10.967889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:10.967889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:10.967889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:10.967889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:10.967889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:10.967889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:10.967889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:10.967889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:16:10.967889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:16:10.967889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:16:10.968854 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:16:10.968854 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:10.968854 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:10.968854 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:10.968854 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:10.968854 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:10.968854 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:10.968854 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:10.968854 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:10.968854 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:10.968854 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:10.968854 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:10.968854 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:10.968854 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:10.968854 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:10.968854 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:10.968854 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:10.969779 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:16:10.969779 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:16:10.969779 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:10.969779 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:10.977685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 441/892] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:16:10.979224 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5032cc-1caf-4214-a031-49020ae975cf None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:10.988341 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba49cf2-813e-4f78-846f-3d7f09e8e5c5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:11.008733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:11.034839 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688269f-f40e-4db0-a812-028132e5acaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:11.035616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688269f-f40e-4db0-a812-028132e5acaa None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:11.035912 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688269f-f40e-4db0-a812-028132e5acaa None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:11.036199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688269f-f40e-4db0-a812-028132e5acaa None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:11.037514 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688269f-f40e-4db0-a812-028132e5acaa None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:11.037882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688269f-f40e-4db0-a812-028132e5acaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:11.039378 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688269f-f40e-4db0-a812-028132e5acaa None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:11.050275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 452/893] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 30 18:16:10 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:11.051230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688269f-f40e-4db0-a812-028132e5acaa None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:11.062824 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:11.100615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5032cc-1caf-4214-a031-49020ae975cf None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:11.139700 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01846e92-fd70-4977-bfd3-56eb1d18662c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:11.141220 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01846e92-fd70-4977-bfd3-56eb1d18662c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:11.142187 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01846e92-fd70-4977-bfd3-56eb1d18662c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:11.142948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01846e92-fd70-4977-bfd3-56eb1d18662c None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:11.143882 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01846e92-fd70-4977-bfd3-56eb1d18662c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:11.144635 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01846e92-fd70-4977-bfd3-56eb1d18662c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:11.147894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01846e92-fd70-4977-bfd3-56eb1d18662c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:11.165122 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688269f-f40e-4db0-a812-028132e5acaa None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:11.166436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 442/894] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:11.174654 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01846e92-fd70-4977-bfd3-56eb1d18662c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:11.214626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d3938-995b-4e14-8d02-ac84a3298e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:11.215766 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d3938-995b-4e14-8d02-ac84a3298e6f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:11.216387 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d3938-995b-4e14-8d02-ac84a3298e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:11.216788 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d3938-995b-4e14-8d02-ac84a3298e6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:11.218484 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d3938-995b-4e14-8d02-ac84a3298e6f None admin] RBAC: Adding query filter params (interface=public, service_id=f6cceadf40bf434bb3b7bc4b43596e62, region_id=RegionOne) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:11.219126 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d3938-995b-4e14-8d02-ac84a3298e6f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:11.221997 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d3938-995b-4e14-8d02-ac84a3298e6f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:11.233749 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 453/895] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 30 18:16:11 2022] GET /identity/v3/endpoints?service_id=f6cceadf40bf434bb3b7bc4b43596e62&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:11.234756 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d3938-995b-4e14-8d02-ac84a3298e6f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:11.305566 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:11.319601 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01846e92-fd70-4977-bfd3-56eb1d18662c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:11.356563 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:11.368424 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536181f6-7caa-459b-a156-b94f4b9cea29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:11.369570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536181f6-7caa-459b-a156-b94f4b9cea29 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:11.369869 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536181f6-7caa-459b-a156-b94f4b9cea29 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:11.370141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536181f6-7caa-459b-a156-b94f4b9cea29 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:11.378086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536181f6-7caa-459b-a156-b94f4b9cea29 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:11.379614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536181f6-7caa-459b-a156-b94f4b9cea29 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:11.393786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d3938-995b-4e14-8d02-ac84a3298e6f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:11.397144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-536181f6-7caa-459b-a156-b94f4b9cea29 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 18:16:11.397144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:11.397144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:11.397144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:11.397144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:11.397144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:11.397144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:11.397144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:11.397144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:11.397144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:11.397144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:11.397144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:11.397144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:16:11.397144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:16:11.397144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:16:11.397144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:16:11.398191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:11.398191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:11.398191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:16:11.398191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:16:11.398191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:16:11.398191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:16:11.398191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:11.398191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:11.398191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:11.398191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:11.398191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:11.398191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:11.398191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:11.398191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:11.398191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:11.398191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:11.398191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:11.398191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:11.399148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:11.399148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:11.399148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:16:11.399148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:16:11.399148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:16:11.399148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:16:11.399148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:16:11.399148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:16:11.399148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 18:16:11.399148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:11.399148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 443/896] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 30 18:16:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:11.400084 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f73c23-b70e-41bd-b9c4-9b61f76de20c None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:11.400084 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f73c23-b70e-41bd-b9c4-9b61f76de20c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:11.400341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f73c23-b70e-41bd-b9c4-9b61f76de20c None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:11.402081 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 444/897] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:11.406497 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f73c23-b70e-41bd-b9c4-9b61f76de20c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:11.446986 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544dd6f0-544e-4dbc-9ba7-2d2ff28929cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:11.447823 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544dd6f0-544e-4dbc-9ba7-2d2ff28929cb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:11.448447 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544dd6f0-544e-4dbc-9ba7-2d2ff28929cb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:11.448737 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544dd6f0-544e-4dbc-9ba7-2d2ff28929cb None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:11.449604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544dd6f0-544e-4dbc-9ba7-2d2ff28929cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:11.451017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544dd6f0-544e-4dbc-9ba7-2d2ff28929cb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:11.464108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 454/898] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 30 18:16:11 2022] GET /identity/v3/services => generated 1268 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:16:11.464108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32de3eb-d3ff-4d3e-a6e2-4a8140221a1a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:11.464108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32de3eb-d3ff-4d3e-a6e2-4a8140221a1a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:11.464108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32de3eb-d3ff-4d3e-a6e2-4a8140221a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:11.483244 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b32de3eb-d3ff-4d3e-a6e2-4a8140221a1a None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:11.485375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b32de3eb-d3ff-4d3e-a6e2-4a8140221a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:11.548804 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f73c23-b70e-41bd-b9c4-9b61f76de20c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:11.596543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 455/899] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:11 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:11.601624 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c101de-ab5e-4beb-9f4c-1a208253cf4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:11.602356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c101de-ab5e-4beb-9f4c-1a208253cf4b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:11.602640 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c101de-ab5e-4beb-9f4c-1a208253cf4b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:11.602918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c101de-ab5e-4beb-9f4c-1a208253cf4b None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:11.603792 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c101de-ab5e-4beb-9f4c-1a208253cf4b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:11.604103 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c101de-ab5e-4beb-9f4c-1a208253cf4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:11.605828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c101de-ab5e-4beb-9f4c-1a208253cf4b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:11.617115 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32de3eb-d3ff-4d3e-a6e2-4a8140221a1a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:11.637325 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 445/900] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Jun 30 18:16:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:11.638636 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c101de-ab5e-4beb-9f4c-1a208253cf4b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:11.697490 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:11.743839 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:11.808313 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32de3eb-d3ff-4d3e-a6e2-4a8140221a1a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:11.848826 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c101de-ab5e-4beb-9f4c-1a208253cf4b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:11.871941 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b41fe5-8da1-4964-bdc8-e350cdf40289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:11.873442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b41fe5-8da1-4964-bdc8-e350cdf40289 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:11.873973 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b41fe5-8da1-4964-bdc8-e350cdf40289 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:11.874632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b41fe5-8da1-4964-bdc8-e350cdf40289 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:11.875903 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b41fe5-8da1-4964-bdc8-e350cdf40289 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:11.877978 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b41fe5-8da1-4964-bdc8-e350cdf40289 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:11.889419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65cd552-605f-4a4e-84d6-c53acf295fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:11.890706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cd552-605f-4a4e-84d6-c53acf295fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:11.891409 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cd552-605f-4a4e-84d6-c53acf295fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:11.892094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cd552-605f-4a4e-84d6-c53acf295fd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:11.893349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-06b41fe5-8da1-4964-bdc8-e350cdf40289 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 18:16:11.893349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:11.893349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:16:11.893349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:16:11.893349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:16:11.893349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:16:11.893349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:16:11.893349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:11.893349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 18:16:11.893349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:11.893349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:16:11.893349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:11.893349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:11.893349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:11.893349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:11.893349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:11.893349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:11.894365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:11.894365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:11.894365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:11.894365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:11.894365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:11.894365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:11.894365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:16:11.894365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:16:11.894365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:16:11.894365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:16:11.894365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:11.894365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:11.894365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:11.894365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:11.894365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:11.894365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:11.895204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:11.895204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:11.895204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:11.895204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:11.895204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:11.895204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:11.895204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:11.895204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:11.895204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:11.895204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:11.895204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:16:11.895204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:11.895204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 18:16:11.895204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:11.899581 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 456/901] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 30 18:16:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:16:11.901896 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b41fe5-8da1-4964-bdc8-e350cdf40289 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:12.009246 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65cd552-605f-4a4e-84d6-c53acf295fd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:12.012173 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65cd552-605f-4a4e-84d6-c53acf295fd3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:12.035045 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b41fe5-8da1-4964-bdc8-e350cdf40289 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:12.040493 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:12.072062 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:12.088798 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866615d5-1424-4ace-a205-51a0a7eadb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:12.089646 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866615d5-1424-4ace-a205-51a0a7eadb54 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:12.089938 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866615d5-1424-4ace-a205-51a0a7eadb54 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:12.090218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866615d5-1424-4ace-a205-51a0a7eadb54 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:12.091064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866615d5-1424-4ace-a205-51a0a7eadb54 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:12.091683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866615d5-1424-4ace-a205-51a0a7eadb54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:12.092840 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866615d5-1424-4ace-a205-51a0a7eadb54 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:12.098271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 457/902] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 30 18:16:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:12.104255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e8f6b-e8ed-4b7e-92ae-6817e7bb2dbe None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:12.104546 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e8f6b-e8ed-4b7e-92ae-6817e7bb2dbe None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:12.104827 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e8f6b-e8ed-4b7e-92ae-6817e7bb2dbe None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:12.110216 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 458/903] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:12.110216 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e8f6b-e8ed-4b7e-92ae-6817e7bb2dbe None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:12.138855 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 446/904] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:16:11 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 18:16:12.143037 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba871d-8384-4794-b41f-ceb96f366ad1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:12.143328 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba871d-8384-4794-b41f-ceb96f366ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:12.143612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba871d-8384-4794-b41f-ceb96f366ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:12.162849 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-57ba871d-8384-4794-b41f-ceb96f366ad1 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:12.164087 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-57ba871d-8384-4794-b41f-ceb96f366ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:12.249639 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 447/905] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:12 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:12.249639 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba871d-8384-4794-b41f-ceb96f366ad1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:12.266514 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e8f6b-e8ed-4b7e-92ae-6817e7bb2dbe None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:12.321496 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7224e-e060-453b-8e44-a398e461da90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:12.322617 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7224e-e060-453b-8e44-a398e461da90 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:12.322910 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7224e-e060-453b-8e44-a398e461da90 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:12.323188 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7224e-e060-453b-8e44-a398e461da90 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:12.323916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7224e-e060-453b-8e44-a398e461da90 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:12.324230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7224e-e060-453b-8e44-a398e461da90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:12.325653 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7224e-e060-453b-8e44-a398e461da90 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:12.330713 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 459/906] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 30 18:16:12 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:12.335562 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7224e-e060-453b-8e44-a398e461da90 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:12.373378 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:12.393235 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:12.419903 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba871d-8384-4794-b41f-ceb96f366ad1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:12.472360 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6c679-fd7d-4da1-af05-3a46be7fffcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:12.473693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6c679-fd7d-4da1-af05-3a46be7fffcb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:12.474225 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6c679-fd7d-4da1-af05-3a46be7fffcb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:12.475378 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6c679-fd7d-4da1-af05-3a46be7fffcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:12.492558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6c679-fd7d-4da1-af05-3a46be7fffcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:12.495434 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6c679-fd7d-4da1-af05-3a46be7fffcb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:12.518781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7224e-e060-453b-8e44-a398e461da90 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:12.521715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f8e6c679-fd7d-4da1-af05-3a46be7fffcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:12.521715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:12.521715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:12.521715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:12.521715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:12.521715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:12.521715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:12.521715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:12.521715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:12.521715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:12.521715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:12.521715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:12.521715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:16:12.521715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:16:12.521715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:16:12.523199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:16:12.523199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:12.523199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:12.523199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:12.523199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:12.523199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:12.523199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:12.523199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:12.523199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:12.523199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:12.523199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:12.523199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:12.523199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:12.523199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:12.523199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:12.523199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:12.523199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:12.524154 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:16:12.524154 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:16:12.524154 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:12.524154 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:12.526148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 448/907] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:12.540047 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6c679-fd7d-4da1-af05-3a46be7fffcb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:12.573768 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad9fbf-0629-460d-8e4b-c564523b8a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:12.574615 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad9fbf-0629-460d-8e4b-c564523b8a73 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:12.574911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad9fbf-0629-460d-8e4b-c564523b8a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:12.576679 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad9fbf-0629-460d-8e4b-c564523b8a73 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:12.576679 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad9fbf-0629-460d-8e4b-c564523b8a73 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:12.578010 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad9fbf-0629-460d-8e4b-c564523b8a73 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:12.596108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-54ad9fbf-0629-460d-8e4b-c564523b8a73 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 18:16:12.596108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:12.596108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:16:12.596108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:16:12.596108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:16:12.596108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:16:12.596108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:16:12.596108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:12.596108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 18:16:12.596108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:12.596108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:16:12.596108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:12.596108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:12.596108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:12.596108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:12.596108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:12.598321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:12.598321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:12.598321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:12.598321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:12.598321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:12.598321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:12.598321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:12.598321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:16:12.598321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:16:12.598321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:16:12.598321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:16:12.598321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:12.598321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:12.598321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:12.598321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:12.598321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:12.600589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:12.600589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:12.600589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:12.600589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:12.600589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:12.600589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:12.600589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:12.600589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:12.600589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:12.600589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:12.600589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:12.600589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:16:12.600589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:12.600589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 18:16:12.600589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:12.600589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 460/908] 10.222.0.33 () {56 vars in 1132 bytes} [Thu Jun 30 18:16:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:12.607210 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad9fbf-0629-460d-8e4b-c564523b8a73 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:12.713505 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6c679-fd7d-4da1-af05-3a46be7fffcb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:12.738188 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:12.745977 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad9fbf-0629-460d-8e4b-c564523b8a73 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:12.750354 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:12.763811 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c0f21-dfdc-4c68-8edc-ff713012770f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:12.765163 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c0f21-dfdc-4c68-8edc-ff713012770f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:12.765993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c0f21-dfdc-4c68-8edc-ff713012770f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:12.766438 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c0f21-dfdc-4c68-8edc-ff713012770f None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:12.767518 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c0f21-dfdc-4c68-8edc-ff713012770f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:12.768035 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c0f21-dfdc-4c68-8edc-ff713012770f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:12.769751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c0f21-dfdc-4c68-8edc-ff713012770f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:12.790066 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 449/909] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:12.793271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c0f21-dfdc-4c68-8edc-ff713012770f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:12.796596 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63b321-f178-41a4-b0ff-84e11bb3417c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:12.797551 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63b321-f178-41a4-b0ff-84e11bb3417c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:12.797980 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63b321-f178-41a4-b0ff-84e11bb3417c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:12.798389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63b321-f178-41a4-b0ff-84e11bb3417c None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:12.799789 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e63b321-f178-41a4-b0ff-84e11bb3417c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:12.799789 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e63b321-f178-41a4-b0ff-84e11bb3417c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:12.801322 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e63b321-f178-41a4-b0ff-84e11bb3417c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:12.808901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 461/910] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 30 18:16:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:12.814593 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63b321-f178-41a4-b0ff-84e11bb3417c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:12.927084 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c0f21-dfdc-4c68-8edc-ff713012770f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:12.947256 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63b321-f178-41a4-b0ff-84e11bb3417c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:12.968558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3108b0-03ab-4ac6-81d2-b2e9de613606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:12.970093 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3108b0-03ab-4ac6-81d2-b2e9de613606 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:12.970570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3108b0-03ab-4ac6-81d2-b2e9de613606 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:12.970998 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3108b0-03ab-4ac6-81d2-b2e9de613606 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:12.979227 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3108b0-03ab-4ac6-81d2-b2e9de613606 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:12.981652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3108b0-03ab-4ac6-81d2-b2e9de613606 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:12.995748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5c271-7a6d-4426-93ed-fd514bd9e982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:12.996766 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5c271-7a6d-4426-93ed-fd514bd9e982 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:12.997202 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5c271-7a6d-4426-93ed-fd514bd9e982 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:12.997775 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5c271-7a6d-4426-93ed-fd514bd9e982 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:12.998832 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5c271-7a6d-4426-93ed-fd514bd9e982 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:12.999297 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5c271-7a6d-4426-93ed-fd514bd9e982 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:12.999976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-5f3108b0-03ab-4ac6-81d2-b2e9de613606 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:16:12.999976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:12.999976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:12.999976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:12.999976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:12.999976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:12.999976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:12.999976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:12.999976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:12.999976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:12.999976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:12.999976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:12.999976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:16:12.999976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:16:12.999976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:16:13.006668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:16:13.006668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:13.006668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:13.006668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:13.006668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:13.006668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:13.006668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:13.006668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:13.006668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:13.006668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:13.006668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:13.006668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:13.006668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:13.006668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:13.006668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:13.006668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:13.006668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:13.007989 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:16:13.007989 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:16:13.007989 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:16:13.007989 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:16:13.007989 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:16:13.007989 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:16:13.007989 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:16:13.007989 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:13.007989 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 450/911] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:16:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:13.010384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5c271-7a6d-4426-93ed-fd514bd9e982 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:13.012444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3108b0-03ab-4ac6-81d2-b2e9de613606 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:13.019337 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 462/912] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 30 18:16:12 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:13.028467 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5c271-7a6d-4426-93ed-fd514bd9e982 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:13.074511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:13.109535 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:13.187329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3108b0-03ab-4ac6-81d2-b2e9de613606 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:13.211943 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5c271-7a6d-4426-93ed-fd514bd9e982 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:13.246674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bfc0e2-1ae3-4753-8591-0f94378e6deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:13.248019 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfc0e2-1ae3-4753-8591-0f94378e6deb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:13.248571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfc0e2-1ae3-4753-8591-0f94378e6deb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:13.248960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfc0e2-1ae3-4753-8591-0f94378e6deb None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:13.250261 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bfc0e2-1ae3-4753-8591-0f94378e6deb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:13.251004 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bfc0e2-1ae3-4753-8591-0f94378e6deb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:13.252675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bfc0e2-1ae3-4753-8591-0f94378e6deb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:13.276132 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1bf34-5463-4e8c-9551-755841fdee04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:13.276986 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1bf34-5463-4e8c-9551-755841fdee04 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:13.277434 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1bf34-5463-4e8c-9551-755841fdee04 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:13.277786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1bf34-5463-4e8c-9551-755841fdee04 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:13.278776 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 451/913] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:16:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:13.284212 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1bf34-5463-4e8c-9551-755841fdee04 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:13.285459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bfc0e2-1ae3-4753-8591-0f94378e6deb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:13.285882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1bf34-5463-4e8c-9551-755841fdee04 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:13.326109 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 463/914] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 30 18:16:13 2022] POST /identity/v3/endpoints => generated 347 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:16:13.427539 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bfc0e2-1ae3-4753-8591-0f94378e6deb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:13.440459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:13.483801 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f1248-9a4c-4c9a-ae6c-9faadf9a9faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:13.484923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f1248-9a4c-4c9a-ae6c-9faadf9a9faf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:13.485526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f1248-9a4c-4c9a-ae6c-9faadf9a9faf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:13.486050 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f1248-9a4c-4c9a-ae6c-9faadf9a9faf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:13.487132 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01f1248-9a4c-4c9a-ae6c-9faadf9a9faf None admin] RBAC: Adding query filter params (role.id=fca7293e63b14558bb206aaac4e5d1ca, scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b, user.id=4ddcf6306e704fe9ad5ed8779178ae67) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:13.487689 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01f1248-9a4c-4c9a-ae6c-9faadf9a9faf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:13.489538 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01f1248-9a4c-4c9a-ae6c-9faadf9a9faf None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:13.497805 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 452/915] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:16:13 2022] GET /identity/v3/role_assignments?user.id=4ddcf6306e704fe9ad5ed8779178ae67&scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b&role.id=fca7293e63b14558bb206aaac4e5d1ca => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:14.249271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4d91f-fec9-4792-b739-385c820d5340 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:14.250136 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4d91f-fec9-4792-b739-385c820d5340 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:14.250307 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4d91f-fec9-4792-b739-385c820d5340 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:14.253415 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 464/916] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:14.263157 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9db301-d677-40a4-aed5-0e3e6cc3c925 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:14.263677 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9db301-d677-40a4-aed5-0e3e6cc3c925 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:14.264183 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9db301-d677-40a4-aed5-0e3e6cc3c925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:14.295149 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6b9db301-d677-40a4-aed5-0e3e6cc3c925 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:14.297412 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6b9db301-d677-40a4-aed5-0e3e6cc3c925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:14.422173 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 453/917] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:14 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:14.446501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e4d91f-fec9-4792-b739-385c820d5340 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:14.537858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:14.561485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e4d91f-fec9-4792-b739-385c820d5340 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:14.610905 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a8de2-7774-4a0e-8e46-800bfa931fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:14.612021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a8de2-7774-4a0e-8e46-800bfa931fcb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:14.612500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a8de2-7774-4a0e-8e46-800bfa931fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:14.612962 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a8de2-7774-4a0e-8e46-800bfa931fcb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:14.614324 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a8de2-7774-4a0e-8e46-800bfa931fcb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:14.617607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a8de2-7774-4a0e-8e46-800bfa931fcb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:14.641643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-3f4a8de2-7774-4a0e-8e46-800bfa931fcb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 18:16:14.641643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:14.641643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:16:14.641643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:16:14.641643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:16:14.641643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:16:14.641643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:16:14.641643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:14.641643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 18:16:14.641643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:14.641643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:16:14.641643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:14.641643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:14.641643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:14.641643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:14.641643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:14.641643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:14.642734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:14.642734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:14.642734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:14.642734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:14.642734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:14.642734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:14.642734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:16:14.642734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:16:14.642734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:16:14.642734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:16:14.642734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:14.642734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:14.642734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:14.642734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:14.642734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:14.642734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:14.643574 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:14.643574 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:14.643574 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:14.643574 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:14.643574 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:14.643574 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:14.643574 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:14.643574 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:14.643574 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:14.643574 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:14.643574 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:16:14.643574 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:14.643574 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 18:16:14.643574 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:14.644836 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 465/918] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 30 18:16:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:14.654947 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9db301-d677-40a4-aed5-0e3e6cc3c925 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:14.739838 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1da36-d087-4f33-8bd8-b543d346170d None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:14.740840 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1da36-d087-4f33-8bd8-b543d346170d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:14.741349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1da36-d087-4f33-8bd8-b543d346170d None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:14.745414 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 466/919] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:14.752193 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759d6d9-6552-4943-b468-9f0d46f574ac None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:14.752827 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759d6d9-6552-4943-b468-9f0d46f574ac None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:14.753338 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759d6d9-6552-4943-b468-9f0d46f574ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:14.774138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d759d6d9-6552-4943-b468-9f0d46f574ac None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:14.776081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d759d6d9-6552-4943-b468-9f0d46f574ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:14.857185 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:14.876797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9db301-d677-40a4-aed5-0e3e6cc3c925 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:14.935440 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 467/920] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:14 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:16:14.974964 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759d6d9-6552-4943-b468-9f0d46f574ac None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:14.987095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec9cb5-eb6c-419d-9e96-492a3e7a5df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:14.989366 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec9cb5-eb6c-419d-9e96-492a3e7a5df1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:14.990083 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec9cb5-eb6c-419d-9e96-492a3e7a5df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:14.991421 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec9cb5-eb6c-419d-9e96-492a3e7a5df1 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:14.992371 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec9cb5-eb6c-419d-9e96-492a3e7a5df1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:14.993091 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec9cb5-eb6c-419d-9e96-492a3e7a5df1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:14.996261 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec9cb5-eb6c-419d-9e96-492a3e7a5df1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:15.015145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 454/921] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 30 18:16:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:15.016800 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec9cb5-eb6c-419d-9e96-492a3e7a5df1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:15.127175 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:15.185202 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759d6d9-6552-4943-b468-9f0d46f574ac None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:15.222659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec9cb5-eb6c-419d-9e96-492a3e7a5df1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:15.227258 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdfd353-279c-4506-a7ac-a87499942fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:15.228189 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfd353-279c-4506-a7ac-a87499942fc8 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:15.228706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfd353-279c-4506-a7ac-a87499942fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:15.229289 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfd353-279c-4506-a7ac-a87499942fc8 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:15.230487 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdfd353-279c-4506-a7ac-a87499942fc8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:15.232100 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdfd353-279c-4506-a7ac-a87499942fc8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:15.265731 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 468/922] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 30 18:16:14 2022] POST /identity/v3/services => generated 241 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:16:15.357818 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec7e93-8949-4acc-bd3a-7c3811e19732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:15.359001 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec7e93-8949-4acc-bd3a-7c3811e19732 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:15.359526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec7e93-8949-4acc-bd3a-7c3811e19732 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:15.360040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec7e93-8949-4acc-bd3a-7c3811e19732 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:15.366023 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ec7e93-8949-4acc-bd3a-7c3811e19732 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:15.366784 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ec7e93-8949-4acc-bd3a-7c3811e19732 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:15.368589 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ec7e93-8949-4acc-bd3a-7c3811e19732 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:15.375641 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 455/923] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 30 18:16:15 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:15.382166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdfd353-279c-4506-a7ac-a87499942fc8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:15.513208 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdfd353-279c-4506-a7ac-a87499942fc8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:15.552874 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdec4d-d447-41c6-94e2-f5164478c9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:15.553776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdec4d-d447-41c6-94e2-f5164478c9ce None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:15.554068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdec4d-d447-41c6-94e2-f5164478c9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:15.554540 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdec4d-d447-41c6-94e2-f5164478c9ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:15.555301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdec4d-d447-41c6-94e2-f5164478c9ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:15.556744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdec4d-d447-41c6-94e2-f5164478c9ce None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:15.560608 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:15.581871 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 469/924] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 30 18:16:15 2022] POST /identity/v3/endpoints => generated 346 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:16:15.922151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19357b1b-ccfc-4e56-b0f0-e951ed47f56e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:15.922689 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19357b1b-ccfc-4e56-b0f0-e951ed47f56e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:15.923000 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19357b1b-ccfc-4e56-b0f0-e951ed47f56e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:15.924787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 456/925] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:15.931986 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617e076-e1cb-4346-b63b-ee597a56901d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:15.932427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617e076-e1cb-4346-b63b-ee597a56901d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:15.932814 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617e076-e1cb-4346-b63b-ee597a56901d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:15.951585 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4617e076-e1cb-4346-b63b-ee597a56901d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:15.952881 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4617e076-e1cb-4346-b63b-ee597a56901d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:16.045456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 470/926] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:16.063563 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecb237-27b9-4528-a7d6-d50699ff3e4e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:16.063899 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecb237-27b9-4528-a7d6-d50699ff3e4e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:16.064132 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecb237-27b9-4528-a7d6-d50699ff3e4e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:16.065848 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 457/927] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:16.071806 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb979c23-1f5f-4005-94cf-13c911dc5921 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:16.072110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb979c23-1f5f-4005-94cf-13c911dc5921 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:16.072308 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ecb237-27b9-4528-a7d6-d50699ff3e4e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:16.072414 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb979c23-1f5f-4005-94cf-13c911dc5921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:16.095011 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cb979c23-1f5f-4005-94cf-13c911dc5921 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:16.096361 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cb979c23-1f5f-4005-94cf-13c911dc5921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:16.109817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:16.210791 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ecb237-27b9-4528-a7d6-d50699ff3e4e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:16.221342 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 471/928] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:16.247180 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb979c23-1f5f-4005-94cf-13c911dc5921 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:16.274641 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807be5f-e275-4f99-84e8-d770e4fb1e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:16.275789 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807be5f-e275-4f99-84e8-d770e4fb1e24 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:16.276856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807be5f-e275-4f99-84e8-d770e4fb1e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:16.277330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807be5f-e275-4f99-84e8-d770e4fb1e24 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:16.294735 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e807be5f-e275-4f99-84e8-d770e4fb1e24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:16:16.306076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:16.308348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e807be5f-e275-4f99-84e8-d770e4fb1e24 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 18:16:16.308348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:16.308348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:16.308348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:16.308348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:16.308348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:16.308348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:16.308348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:16.308348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:16.308348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:16.308348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:16.308348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:16.308348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:16:16.308348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:16:16.308348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:16:16.308348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:16:16.309337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:16:16.309337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:16:16.309337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:16:16.309337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:16:16.309337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:16.309337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:16.309337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:16.309337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:16.309337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:16.309337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:16.309337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:16.309337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:16.309337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:16.309337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:16.309337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:16.309337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:16.310214 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:16.310214 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:16.310214 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:16.310214 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:16.310214 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:16:16.310214 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:16:16.310214 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:16:16.310214 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:16:16.310214 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:16:16.310214 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:16:16.310214 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 18:16:16.310214 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:16.313856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 458/929] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Jun 30 18:16:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:16:16.318419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807be5f-e275-4f99-84e8-d770e4fb1e24 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:16.377908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb979c23-1f5f-4005-94cf-13c911dc5921 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:16.424665 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48625946-8079-45a2-b311-f183a3ee6f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:16.425884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48625946-8079-45a2-b311-f183a3ee6f88 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:16.426183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48625946-8079-45a2-b311-f183a3ee6f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:16.426465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48625946-8079-45a2-b311-f183a3ee6f88 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:16.427133 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48625946-8079-45a2-b311-f183a3ee6f88 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:16.429216 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48625946-8079-45a2-b311-f183a3ee6f88 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:16.431553 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:16.445081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-48625946-8079-45a2-b311-f183a3ee6f88 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 18:16:16.445081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:16.445081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:16:16.445081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:16:16.445081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:16:16.445081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:16:16.445081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:16:16.445081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:16.445081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 18:16:16.445081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:16.445081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:16:16.445081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:16.445081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:16.445081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:16.445081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:16.445081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:16.445081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:16.447363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:16.447363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:16.447363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:16.447363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:16.447363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:16.447363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:16.447363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:16:16.447363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:16:16.447363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:16:16.447363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:16:16.447363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:16.447363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:16.447363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:16.447363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:16.447363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:16.447363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:16.448578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:16.448578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:16.448578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:16.448578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:16.448578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:16.448578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:16.448578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:16.448578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:16.448578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:16.448578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:16.448578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:16:16.448578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:16.448578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 18:16:16.448578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:16.453145 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 472/930] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:16:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:16:16.453145 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48625946-8079-45a2-b311-f183a3ee6f88 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:16.459625 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807be5f-e275-4f99-84e8-d770e4fb1e24 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:16.508755 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21af07-077a-434a-9571-778ab3fb9c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:16.509344 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21af07-077a-434a-9571-778ab3fb9c89 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:16.509655 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21af07-077a-434a-9571-778ab3fb9c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:16.509938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21af07-077a-434a-9571-778ab3fb9c89 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:16.510695 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21af07-077a-434a-9571-778ab3fb9c89 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:16.511006 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21af07-077a-434a-9571-778ab3fb9c89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:16.512388 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21af07-077a-434a-9571-778ab3fb9c89 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:16.521865 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 459/931] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 30 18:16:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:16.528219 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21af07-077a-434a-9571-778ab3fb9c89 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:16.572715 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48625946-8079-45a2-b311-f183a3ee6f88 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:16.585762 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:16.610525 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b645ef-40a8-481c-9c4f-9a998f0b4823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:16.611359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b645ef-40a8-481c-9c4f-9a998f0b4823 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:16.611749 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b645ef-40a8-481c-9c4f-9a998f0b4823 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:16.612133 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b645ef-40a8-481c-9c4f-9a998f0b4823 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:16.612958 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b645ef-40a8-481c-9c4f-9a998f0b4823 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:16.613373 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b645ef-40a8-481c-9c4f-9a998f0b4823 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:16.615126 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b645ef-40a8-481c-9c4f-9a998f0b4823 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:16.625784 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 473/932] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 30 18:16:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:16.629768 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b645ef-40a8-481c-9c4f-9a998f0b4823 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:16.669694 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21af07-077a-434a-9571-778ab3fb9c89 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:16.720646 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452e872-31b7-401a-9d61-bd65edecb2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:16.721443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452e872-31b7-401a-9d61-bd65edecb2f6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:16.721754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452e872-31b7-401a-9d61-bd65edecb2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:16.722035 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452e872-31b7-401a-9d61-bd65edecb2f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:16.735699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f452e872-31b7-401a-9d61-bd65edecb2f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:16.737476 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f452e872-31b7-401a-9d61-bd65edecb2f6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:16.740756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:16.754653 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b645ef-40a8-481c-9c4f-9a998f0b4823 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:16.757733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f452e872-31b7-401a-9d61-bd65edecb2f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:16.757733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:16.757733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:16.757733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:16.757733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:16.757733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:16.757733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:16.757733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:16.757733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:16.757733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:16.757733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:16.757733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:16.757733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:16:16.757733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:16:16.757733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:16:16.758601 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:16:16.758601 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:16.758601 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:16.758601 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:16.758601 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:16.758601 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:16.758601 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:16.758601 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:16.758601 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:16.758601 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:16.758601 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:16.758601 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:16.758601 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:16.758601 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:16.758601 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:16.758601 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:16.758601 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:16.759578 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:16:16.759578 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:16:16.759578 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:16.759578 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:16.761832 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 460/933] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:16:16.767730 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452e872-31b7-401a-9d61-bd65edecb2f6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:16.796437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650277e-f19a-4f48-9c7f-80923275fd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:16.797504 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650277e-f19a-4f48-9c7f-80923275fd34 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:16.798100 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650277e-f19a-4f48-9c7f-80923275fd34 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:16.798583 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650277e-f19a-4f48-9c7f-80923275fd34 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:16.799490 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650277e-f19a-4f48-9c7f-80923275fd34 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:16.800016 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650277e-f19a-4f48-9c7f-80923275fd34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:16.801625 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650277e-f19a-4f48-9c7f-80923275fd34 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:16.813699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 474/934] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 30 18:16:16 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:16.890109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452e872-31b7-401a-9d61-bd65edecb2f6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:16.932166 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e8158-c045-48b4-9507-df4f69c13535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:16.932859 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e8158-c045-48b4-9507-df4f69c13535 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:16.933127 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e8158-c045-48b4-9507-df4f69c13535 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:16.933399 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e8158-c045-48b4-9507-df4f69c13535 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:16.934230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e8158-c045-48b4-9507-df4f69c13535 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:16.934546 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e8158-c045-48b4-9507-df4f69c13535 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:16.935956 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e8158-c045-48b4-9507-df4f69c13535 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:16.950350 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 461/935] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:16.956121 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650277e-f19a-4f48-9c7f-80923275fd34 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:17.000311 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:17.070722 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650277e-f19a-4f48-9c7f-80923275fd34 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:17.106106 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897cfdb-18fc-4508-bb00-e5f5def1cfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:17.106934 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897cfdb-18fc-4508-bb00-e5f5def1cfa7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:17.107221 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897cfdb-18fc-4508-bb00-e5f5def1cfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:17.107567 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897cfdb-18fc-4508-bb00-e5f5def1cfa7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:17.116670 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897cfdb-18fc-4508-bb00-e5f5def1cfa7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:17.118166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897cfdb-18fc-4508-bb00-e5f5def1cfa7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:17.134276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-8897cfdb-18fc-4508-bb00-e5f5def1cfa7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 18:16:17.134276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:17.134276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:17.134276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:17.134276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:17.134276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:17.134276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:17.134276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:17.134276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:17.134276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:17.134276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:17.134276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:17.134276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:16:17.134276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:16:17.134276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:16:17.134276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:16:17.135343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:17.135343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:17.135343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:16:17.135343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:16:17.135343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:16:17.135343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:16:17.135343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:17.135343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:17.135343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:17.135343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:17.135343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:17.135343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:17.135343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:17.135343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:17.135343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:17.135343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:17.135343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:17.135343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:17.136300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:17.136300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:17.136300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:16:17.136300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:16:17.136300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:16:17.136300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:16:17.136300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:16:17.136300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:16:17.136300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 18:16:17.136300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:17.136300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 475/936] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 30 18:16:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:17.142623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e8158-c045-48b4-9507-df4f69c13535 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:17.222231 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:17.275083 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e8158-c045-48b4-9507-df4f69c13535 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:17.315031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb041206-6bda-445a-b04f-ef4effd4e1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:17.315627 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb041206-6bda-445a-b04f-ef4effd4e1ab None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:17.315901 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb041206-6bda-445a-b04f-ef4effd4e1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:17.316184 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb041206-6bda-445a-b04f-ef4effd4e1ab None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:17.317040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb041206-6bda-445a-b04f-ef4effd4e1ab None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:17.317353 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb041206-6bda-445a-b04f-ef4effd4e1ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:17.318893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb041206-6bda-445a-b04f-ef4effd4e1ab None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:17.339642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 462/937] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Jun 30 18:16:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:17.345195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897cfdb-18fc-4508-bb00-e5f5def1cfa7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:17.475218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:17.482282 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897cfdb-18fc-4508-bb00-e5f5def1cfa7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:17.519016 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f5dd73-75ce-4a37-8174-6e7188ae8e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:17.519813 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5dd73-75ce-4a37-8174-6e7188ae8e8a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:17.520100 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5dd73-75ce-4a37-8174-6e7188ae8e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:17.520383 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5dd73-75ce-4a37-8174-6e7188ae8e8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:17.531994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f5dd73-75ce-4a37-8174-6e7188ae8e8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:17.537737 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f5dd73-75ce-4a37-8174-6e7188ae8e8a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:17.557781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d1f5dd73-75ce-4a37-8174-6e7188ae8e8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:17.557781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:17.557781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:17.557781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:17.557781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:17.557781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:17.557781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:17.557781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:17.557781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:17.557781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:17.557781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:17.557781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:17.557781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:16:17.557781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:16:17.557781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:16:17.558751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:16:17.558751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:17.558751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:17.558751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:17.558751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:17.558751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:17.558751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:17.558751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:17.558751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:17.558751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:17.558751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:17.558751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:17.558751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:17.558751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:17.558751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:17.558751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:17.558751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:17.559643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:16:17.559643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:16:17.559643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:17.559643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:17.561737 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 476/938] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:17.567152 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb041206-6bda-445a-b04f-ef4effd4e1ab None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:17.642657 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576cdef8-4f58-4d25-aeb5-bfe4929804de None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:17.643348 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576cdef8-4f58-4d25-aeb5-bfe4929804de None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:17.643844 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576cdef8-4f58-4d25-aeb5-bfe4929804de None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:17.645770 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 477/939] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:16:17.652178 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e66056-f721-4b0e-a1d6-fa11ada8268a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:17.652468 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e66056-f721-4b0e-a1d6-fa11ada8268a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:17.652753 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e66056-f721-4b0e-a1d6-fa11ada8268a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:17.669967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f0e66056-f721-4b0e-a1d6-fa11ada8268a None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:17.671427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f0e66056-f721-4b0e-a1d6-fa11ada8268a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:17.679088 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb041206-6bda-445a-b04f-ef4effd4e1ab None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:17.693941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:17.721178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f83fd05-678f-4df0-b0e6-1ac7361b43cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:17.722025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83fd05-678f-4df0-b0e6-1ac7361b43cb None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:17.722405 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83fd05-678f-4df0-b0e6-1ac7361b43cb None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:17.722778 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83fd05-678f-4df0-b0e6-1ac7361b43cb None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:17.724121 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f83fd05-678f-4df0-b0e6-1ac7361b43cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:17.724793 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f83fd05-678f-4df0-b0e6-1ac7361b43cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:17.726348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f83fd05-678f-4df0-b0e6-1ac7361b43cb None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:17.746814 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 463/940] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:17.747880 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f83fd05-678f-4df0-b0e6-1ac7361b43cb None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:17.761458 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 478/941] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:16:17.784445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e66056-f721-4b0e-a1d6-fa11ada8268a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:17.866872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f83fd05-678f-4df0-b0e6-1ac7361b43cb None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:17.893795 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:17.908732 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ac14d-2db4-4186-9bdf-41ea5b69a422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:17.909532 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ac14d-2db4-4186-9bdf-41ea5b69a422 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:17.909899 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ac14d-2db4-4186-9bdf-41ea5b69a422 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:17.910178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ac14d-2db4-4186-9bdf-41ea5b69a422 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:17.918445 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ac14d-2db4-4186-9bdf-41ea5b69a422 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:17.918770 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e66056-f721-4b0e-a1d6-fa11ada8268a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:17.919905 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ac14d-2db4-4186-9bdf-41ea5b69a422 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:17.936145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-472ac14d-2db4-4186-9bdf-41ea5b69a422 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:16:17.936145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:17.936145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:17.936145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:17.936145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:17.936145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:17.936145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:17.936145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:17.936145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:17.936145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:17.936145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:17.936145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:17.936145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:16:17.936145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:16:17.936145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:16:17.937517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:16:17.937517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:17.937517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:17.937517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:17.937517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:17.937517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:17.937517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:17.937517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:17.937517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:17.937517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:17.937517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:17.937517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:17.937517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:17.937517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:17.937517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:17.937517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:17.937517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:17.938458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:16:17.938458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:16:17.938458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:16:17.938458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:16:17.938458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:16:17.938458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:16:17.938458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:16:17.938458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:17.941920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 464/942] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:16:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:17.944122 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ac14d-2db4-4186-9bdf-41ea5b69a422 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:17.963221 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ea10d-a749-43f5-9fc6-4b5eb90ee45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:17.964042 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ea10d-a749-43f5-9fc6-4b5eb90ee45d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:17.964331 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ea10d-a749-43f5-9fc6-4b5eb90ee45d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:17.964688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ea10d-a749-43f5-9fc6-4b5eb90ee45d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:17.965265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ea10d-a749-43f5-9fc6-4b5eb90ee45d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:17.966675 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ea10d-a749-43f5-9fc6-4b5eb90ee45d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:17.980614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-603ea10d-a749-43f5-9fc6-4b5eb90ee45d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 18:16:17.980614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:17.980614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:16:17.980614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:16:17.980614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:16:17.980614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:16:17.980614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:16:17.980614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:17.980614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 18:16:17.980614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:17.980614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:16:17.980614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:17.980614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:17.980614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:17.980614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:17.980614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:17.980614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:17.981697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:17.981697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:17.981697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:17.981697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:17.981697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:17.981697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:17.981697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:16:17.981697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:16:17.981697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:16:17.981697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:16:17.981697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:17.981697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:17.981697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:17.981697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:17.981697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:17.981697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:17.982587 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:17.982587 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:17.982587 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:17.982587 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:17.982587 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:17.982587 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:17.982587 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:17.982587 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:17.982587 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:17.982587 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:17.982587 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:16:17.982587 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:17.982587 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 18:16:17.982587 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:17.982587 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 479/943] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 30 18:16:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:16:17.983395 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e18cec-a15d-43f9-8e30-9c03eb43f460 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:17.983395 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e18cec-a15d-43f9-8e30-9c03eb43f460 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:17.984173 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e18cec-a15d-43f9-8e30-9c03eb43f460 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:17.985006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 480/944] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:17.990764 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e18cec-a15d-43f9-8e30-9c03eb43f460 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:17.990912 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:18.070685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ac14d-2db4-4186-9bdf-41ea5b69a422 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:18.108983 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e18cec-a15d-43f9-8e30-9c03eb43f460 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:18.110503 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ba194-235f-4530-b782-b6978bcf491f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:18.111160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ba194-235f-4530-b782-b6978bcf491f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:18.111444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ba194-235f-4530-b782-b6978bcf491f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:18.111718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ba194-235f-4530-b782-b6978bcf491f None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:18.112496 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ba194-235f-4530-b782-b6978bcf491f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:18.112805 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ba194-235f-4530-b782-b6978bcf491f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:18.114281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ba194-235f-4530-b782-b6978bcf491f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:18.126666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 465/945] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:16:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:18.127822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92576aac-d8c0-4d49-aabc-6a90d433765f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:18.128109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92576aac-d8c0-4d49-aabc-6a90d433765f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:18.128385 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92576aac-d8c0-4d49-aabc-6a90d433765f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:18.147776 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-92576aac-d8c0-4d49-aabc-6a90d433765f None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:18.148986 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-92576aac-d8c0-4d49-aabc-6a90d433765f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:18.151490 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43dec5c-a2ba-46a0-9b9e-2b5d47b7e7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:18.153121 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43dec5c-a2ba-46a0-9b9e-2b5d47b7e7c9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:18.153612 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43dec5c-a2ba-46a0-9b9e-2b5d47b7e7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:18.154092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43dec5c-a2ba-46a0-9b9e-2b5d47b7e7c9 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:18.154965 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43dec5c-a2ba-46a0-9b9e-2b5d47b7e7c9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:18.155447 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43dec5c-a2ba-46a0-9b9e-2b5d47b7e7c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:18.157392 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43dec5c-a2ba-46a0-9b9e-2b5d47b7e7c9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:18.163701 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 481/946] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 30 18:16:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:18.164490 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43dec5c-a2ba-46a0-9b9e-2b5d47b7e7c9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:18.180150 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:18.220493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 466/947] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:18.221168 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92576aac-d8c0-4d49-aabc-6a90d433765f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:18.253799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:18.288838 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43dec5c-a2ba-46a0-9b9e-2b5d47b7e7c9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:18.329970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ef2c0-9dc5-4082-bad1-92fd28fe5912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:18.331256 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ef2c0-9dc5-4082-bad1-92fd28fe5912 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:18.331757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ef2c0-9dc5-4082-bad1-92fd28fe5912 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:18.332189 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ef2c0-9dc5-4082-bad1-92fd28fe5912 None admin] PATH_INFO: `/v3/projects/8c7e1ee12aa14b7ab13e07822826b25b/users/4ddcf6306e704fe9ad5ed8779178ae67/roles/fca7293e63b14558bb206aaac4e5d1ca` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:18.335657 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ef2c0-9dc5-4082-bad1-92fd28fe5912 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c7e1ee12aa14b7ab13e07822826b25b, user_id=4ddcf6306e704fe9ad5ed8779178ae67, role_id=fca7293e63b14558bb206aaac4e5d1ca)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:18.337517 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ef2c0-9dc5-4082-bad1-92fd28fe5912 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:18.348522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92576aac-d8c0-4d49-aabc-6a90d433765f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:18.357696 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 482/948] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:16:18 2022] PUT /identity/v3/projects/8c7e1ee12aa14b7ab13e07822826b25b/users/4ddcf6306e704fe9ad5ed8779178ae67/roles/fca7293e63b14558bb206aaac4e5d1ca => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:16:18.360278 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ef2c0-9dc5-4082-bad1-92fd28fe5912 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:18.397332 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0b1c3-7e4c-41f5-888a-c46e91399804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:18.398353 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0b1c3-7e4c-41f5-888a-c46e91399804 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:18.398823 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0b1c3-7e4c-41f5-888a-c46e91399804 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:18.399329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0b1c3-7e4c-41f5-888a-c46e91399804 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:18.400442 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd0b1c3-7e4c-41f5-888a-c46e91399804 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:18.400769 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd0b1c3-7e4c-41f5-888a-c46e91399804 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:18.402228 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd0b1c3-7e4c-41f5-888a-c46e91399804 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:18.408738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 467/949] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 30 18:16:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:18.416678 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0b1c3-7e4c-41f5-888a-c46e91399804 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:18.482476 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:18.546737 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ef2c0-9dc5-4082-bad1-92fd28fe5912 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:18.565846 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:18.585102 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0b1c3-7e4c-41f5-888a-c46e91399804 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:18.596924 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f78f1-0ba6-4673-80d3-e4777673baab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:18.597789 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f78f1-0ba6-4673-80d3-e4777673baab None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:18.598016 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f78f1-0ba6-4673-80d3-e4777673baab None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:18.598303 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f78f1-0ba6-4673-80d3-e4777673baab None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:18.599239 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f78f1-0ba6-4673-80d3-e4777673baab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:18.602132 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f78f1-0ba6-4673-80d3-e4777673baab None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:18.603666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: WARNING keystone.server.flask.common [None req-d29f78f1-0ba6-4673-80d3-e4777673baab None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 30 18:16:18.625708 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.sql.core [None req-d29f78f1-0ba6-4673-80d3-e4777673baab None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 18:16:18.625708 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 30 18:16:18.625708 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [parameters: {'id': 'fd81a220b1e04b7abc3735a1cb53dfef', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 30 18:16:18.625708 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80207) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 18:16:18.645427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d29f78f1-0ba6-4673-80d3-e4777673baab None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fd81a220b1e04b7abc3735a1cb53dfef.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fd81a220b1e04b7abc3735a1cb53dfef. Jun 30 18:16:18.645427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:18.645427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 18:16:18.645427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 18:16:18.645427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 18:16:18.645427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 18:16:18.645427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 18:16:18.645427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 18:16:18.645427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 18:16:18.645427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application conn.query(q) Jun 30 18:16:18.645427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 18:16:18.645427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 18:16:18.645427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 18:16:18.647643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application result.read() Jun 30 18:16:18.647643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 18:16:18.647643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 18:16:18.647643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 18:16:18.647643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 18:16:18.647643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 18:16:18.647643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 18:16:18.647643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 18:16:18.647643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 18:16:18.647643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 18:16:18.647643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:18.647643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 18:16:18.647643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:18.647643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:18.647643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 18:16:18.647643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 18:16:18.647643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 30 18:16:18.647643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 30 18:16:18.650705 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 18:16:18.650705 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application next(self.gen) Jun 30 18:16:18.650705 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 18:16:18.650705 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application yield resource Jun 30 18:16:18.650705 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 18:16:18.650705 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application next(self.gen) Jun 30 18:16:18.650705 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 18:16:18.650705 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 18:16:18.650705 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 18:16:18.650705 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 18:16:18.650705 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 18:16:18.650705 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise self.value Jun 30 18:16:18.650705 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 18:16:18.650705 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 18:16:18.650705 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 18:16:18.650705 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application session.commit() Jun 30 18:16:18.650705 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 18:16:18.654384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 18:16:18.654384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 18:16:18.654384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 18:16:18.654384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 18:16:18.654384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.session.flush() Jun 30 18:16:18.654384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 18:16:18.654384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 18:16:18.654384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 18:16:18.654384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 18:16:18.654384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 18:16:18.654384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application compat.raise_( Jun 30 18:16:18.654384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 18:16:18.654384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception Jun 30 18:16:18.654384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 18:16:18.654384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 18:16:18.654384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 18:16:18.654384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 18:16:18.657781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 18:16:18.657781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 18:16:18.657781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 18:16:18.657781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 18:16:18.657781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 30 18:16:18.657781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 30 18:16:18.657781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 18:16:18.657781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 18:16:18.657781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 18:16:18.657781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 18:16:18.657781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 18:16:18.657781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 18:16:18.657781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 18:16:18.657781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 18:16:18.657781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 18:16:18.660985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 18:16:18.660985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 18:16:18.660985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception Jun 30 18:16:18.660985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 18:16:18.660985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 18:16:18.660985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 18:16:18.660985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 18:16:18.660985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 18:16:18.660985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 18:16:18.660985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 18:16:18.660985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application conn.query(q) Jun 30 18:16:18.660985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 18:16:18.660985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 18:16:18.660985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 18:16:18.660985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application result.read() Jun 30 18:16:18.660985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 18:16:18.660985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 18:16:18.664528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 18:16:18.664528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 18:16:18.664528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 18:16:18.664528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 18:16:18.664528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 18:16:18.664528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 18:16:18.664528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 18:16:18.664528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 30 18:16:18.664528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [parameters: {'id': 'fd81a220b1e04b7abc3735a1cb53dfef', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 30 18:16:18.664528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 18:16:18.664528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:18.664528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:16:18.664528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:18.664528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:18.664528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 30 18:16:18.666507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 30 18:16:18.666507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 18:16:18.666507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 18:16:18.666507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 30 18:16:18.666507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:18.666507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:16:18.666507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:18.666507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:18.666507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:18.666507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:18.666507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:18.666507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:18.666507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:18.666507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:18.666507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:18.666507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:18.666507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:18.668894 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:18.668894 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 30 18:16:18.668894 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 30 18:16:18.668894 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:18.668894 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:18.668894 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 30 18:16:18.668894 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 30 18:16:18.668894 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fd81a220b1e04b7abc3735a1cb53dfef. Jun 30 18:16:18.668894 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:18.668894 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 483/950] 10.222.0.33 () {60 vars in 1140 bytes} [Thu Jun 30 18:16:18 2022] POST /identity/v3/projects => generated 262 bytes in 289 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:18.668894 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f78f1-0ba6-4673-80d3-e4777673baab None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:18.676164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a6cfb-af8e-470b-a8fa-367a4019655b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:18.676164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a6cfb-af8e-470b-a8fa-367a4019655b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:18.676164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a6cfb-af8e-470b-a8fa-367a4019655b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:18.676164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a6cfb-af8e-470b-a8fa-367a4019655b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:18.678297 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a6cfb-af8e-470b-a8fa-367a4019655b None admin] RBAC: Adding query filter params (interface=public, service_id=723ffd33dc5e4957b0564e112f9558a8, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:18.678297 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a6cfb-af8e-470b-a8fa-367a4019655b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:18.678297 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a6cfb-af8e-470b-a8fa-367a4019655b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:18.683004 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 468/951] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 30 18:16:18 2022] GET /identity/v3/endpoints?service_id=723ffd33dc5e4957b0564e112f9558a8&interface=public®ion_id=RegionOne => generated 194 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:18.686835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a6cfb-af8e-470b-a8fa-367a4019655b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:18.785721 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f78f1-0ba6-4673-80d3-e4777673baab None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:18.806050 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a6cfb-af8e-470b-a8fa-367a4019655b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:18.826152 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:18.831224 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9301e-5199-40c4-b2e8-867b313a7f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:18.832138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9301e-5199-40c4-b2e8-867b313a7f0f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:18.832589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9301e-5199-40c4-b2e8-867b313a7f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:18.832846 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9301e-5199-40c4-b2e8-867b313a7f0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:18.843349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d9301e-5199-40c4-b2e8-867b313a7f0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:18.845156 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d9301e-5199-40c4-b2e8-867b313a7f0f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:18.850610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276cfd45-641a-4e60-ba28-f1c390199436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:18.852059 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cfd45-641a-4e60-ba28-f1c390199436 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:18.852637 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cfd45-641a-4e60-ba28-f1c390199436 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:18.853128 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cfd45-641a-4e60-ba28-f1c390199436 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:18.854198 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276cfd45-641a-4e60-ba28-f1c390199436 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:18.855798 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276cfd45-641a-4e60-ba28-f1c390199436 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:18.862326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d8d9301e-5199-40c4-b2e8-867b313a7f0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:16:18.862326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:18.862326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:18.862326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:18.862326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:18.862326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:18.862326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:18.862326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:18.862326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:18.862326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:18.862326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:18.862326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:18.862326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:16:18.862326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:16:18.862326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:16:18.863326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:16:18.863326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:18.863326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:18.863326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:18.863326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:18.863326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:18.863326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:18.863326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:18.863326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:18.863326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:18.863326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:18.863326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:18.863326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:18.863326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:18.863326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:18.863326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:18.863326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:18.864283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:16:18.864283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:16:18.864283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:16:18.864283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:16:18.864283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:16:18.864283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:16:18.864283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:16:18.864283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:18.871861 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 484/953] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:16:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:18.872276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 469/953] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 30 18:16:18 2022] GET /identity/v3/services => generated 1497 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:16:18.874198 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9301e-5199-40c4-b2e8-867b313a7f0f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:18.981793 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b332e0-316b-431a-9bbc-fef8228c2956 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:18.982707 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b332e0-316b-431a-9bbc-fef8228c2956 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:18.983191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b332e0-316b-431a-9bbc-fef8228c2956 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:18.985385 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 470/954] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:16:18.993475 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e607e-93b5-42eb-97ae-80775eae4111 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:18.994225 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e607e-93b5-42eb-97ae-80775eae4111 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:18.994796 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e607e-93b5-42eb-97ae-80775eae4111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:19.009997 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9301e-5199-40c4-b2e8-867b313a7f0f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:19.019228 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b40e607e-93b5-42eb-97ae-80775eae4111 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:19.021767 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b40e607e-93b5-42eb-97ae-80775eae4111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:19.041872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:19.072830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8e4f8-183d-4bee-8964-33b485a0a65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:19.074903 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8e4f8-183d-4bee-8964-33b485a0a65b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:19.074903 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8e4f8-183d-4bee-8964-33b485a0a65b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:19.074903 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8e4f8-183d-4bee-8964-33b485a0a65b None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:19.075254 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b8e4f8-183d-4bee-8964-33b485a0a65b None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:19.075506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b8e4f8-183d-4bee-8964-33b485a0a65b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:19.076985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b8e4f8-183d-4bee-8964-33b485a0a65b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:19.100185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 485/955] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 30 18:16:18 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:19.131439 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 471/956] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:16:19.155510 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8e4f8-183d-4bee-8964-33b485a0a65b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:19.214906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:19.276187 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8e4f8-183d-4bee-8964-33b485a0a65b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:19.317078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80c839-bff6-4f84-8c04-e1b2d9362a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:19.317946 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80c839-bff6-4f84-8c04-e1b2d9362a8f None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:19.318555 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80c839-bff6-4f84-8c04-e1b2d9362a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:19.318963 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80c839-bff6-4f84-8c04-e1b2d9362a8f None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:19.319812 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80c839-bff6-4f84-8c04-e1b2d9362a8f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:19.321793 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80c839-bff6-4f84-8c04-e1b2d9362a8f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:19.337981 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 486/957] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 30 18:16:19 2022] POST /identity/v3/services => generated 242 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:16:20.565238 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673fedf-8f8a-497d-a648-bfc05dcb4d6d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:20.566168 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673fedf-8f8a-497d-a648-bfc05dcb4d6d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:20.566604 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673fedf-8f8a-497d-a648-bfc05dcb4d6d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:20.568398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 472/958] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:20.574641 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e92371-b3d1-4718-9d2c-8cece57e91ba None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:20.575552 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e92371-b3d1-4718-9d2c-8cece57e91ba None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:20.576050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e92371-b3d1-4718-9d2c-8cece57e91ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:20.594546 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-82e92371-b3d1-4718-9d2c-8cece57e91ba None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:20.596412 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-82e92371-b3d1-4718-9d2c-8cece57e91ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:20.677331 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 487/959] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:20 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:16:20.702193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673fedf-8f8a-497d-a648-bfc05dcb4d6d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:20.815666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673fedf-8f8a-497d-a648-bfc05dcb4d6d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:20.854298 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa59b8-477d-4bba-b50d-08466325c15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:20.855497 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa59b8-477d-4bba-b50d-08466325c15e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:20.855978 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa59b8-477d-4bba-b50d-08466325c15e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:20.856471 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa59b8-477d-4bba-b50d-08466325c15e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:20.868566 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa59b8-477d-4bba-b50d-08466325c15e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:16:20.877094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-a1aa59b8-477d-4bba-b50d-08466325c15e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 18:16:20.877094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:20.877094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:20.877094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:20.877094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:20.877094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:20.877094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:20.877094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:20.877094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:20.877094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:20.877094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:20.877094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:20.877094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:16:20.877094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:16:20.877094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:16:20.877094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:16:20.878155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:16:20.878155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:16:20.878155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:16:20.878155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:16:20.878155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:20.878155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:20.878155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:20.878155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:20.878155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:20.878155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:20.878155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:20.878155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:20.878155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:20.878155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:20.878155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:20.878155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:20.878990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:20.878990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:20.878990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:20.878990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:20.878990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:16:20.878990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:16:20.878990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:16:20.878990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:16:20.878990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:16:20.878990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:16:20.878990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 18:16:20.878990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:20.885933 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 473/960] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Jun 30 18:16:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:20.888430 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e92371-b3d1-4718-9d2c-8cece57e91ba None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:20.922130 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:21.000764 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e92371-b3d1-4718-9d2c-8cece57e91ba None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:21.040053 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e0bba-a962-47ca-9f75-59fb0e49ee64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:21.040734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e0bba-a962-47ca-9f75-59fb0e49ee64 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:21.041016 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e0bba-a962-47ca-9f75-59fb0e49ee64 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:21.041291 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e0bba-a962-47ca-9f75-59fb0e49ee64 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:21.042069 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e0bba-a962-47ca-9f75-59fb0e49ee64 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:21.042377 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e0bba-a962-47ca-9f75-59fb0e49ee64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:21.043750 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e0bba-a962-47ca-9f75-59fb0e49ee64 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:21.053451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 488/961] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 30 18:16:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:21.059181 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa59b8-477d-4bba-b50d-08466325c15e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:21.075647 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:21.173393 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa59b8-477d-4bba-b50d-08466325c15e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:21.209691 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2b320-5c99-41a9-89b3-cc04b5020dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:21.210904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2b320-5c99-41a9-89b3-cc04b5020dcf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:21.211388 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2b320-5c99-41a9-89b3-cc04b5020dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:21.211883 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2b320-5c99-41a9-89b3-cc04b5020dcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:21.223632 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2b320-5c99-41a9-89b3-cc04b5020dcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:21.226034 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2b320-5c99-41a9-89b3-cc04b5020dcf None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:21.244943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-f8d2b320-5c99-41a9-89b3-cc04b5020dcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:21.244943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:21.244943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:21.244943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:21.244943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:21.244943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:21.244943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:21.244943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:21.244943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:21.244943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:21.244943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:21.244943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:21.244943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:16:21.244943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:16:21.244943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:16:21.245925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:16:21.245925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:21.245925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:21.245925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:21.245925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:21.245925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:21.245925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:21.245925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:21.245925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:21.245925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:21.245925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:21.245925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:21.245925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:21.245925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:21.245925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:21.245925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:21.245925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:21.248248 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:16:21.248248 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:16:21.248248 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:21.248248 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:21.255760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 474/962] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:21.261852 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e0bba-a962-47ca-9f75-59fb0e49ee64 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:21.361814 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:21.372640 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e0bba-a962-47ca-9f75-59fb0e49ee64 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:21.387711 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16c3b5-fd94-4eed-a518-b17165f51207 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:21.388044 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16c3b5-fd94-4eed-a518-b17165f51207 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:21.388272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16c3b5-fd94-4eed-a518-b17165f51207 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:21.391087 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 475/963] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:21.395469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa335c-2c34-42ff-a911-0bed3ad7b5a5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:21.395760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa335c-2c34-42ff-a911-0bed3ad7b5a5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:21.396042 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa335c-2c34-42ff-a911-0bed3ad7b5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:21.416031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-06fa335c-2c34-42ff-a911-0bed3ad7b5a5 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:21.417341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-06fa335c-2c34-42ff-a911-0bed3ad7b5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:21.439021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c709d59-5e26-40a6-8532-fd4b92bb4dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:21.440264 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c709d59-5e26-40a6-8532-fd4b92bb4dbf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:21.440763 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c709d59-5e26-40a6-8532-fd4b92bb4dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:21.441272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c709d59-5e26-40a6-8532-fd4b92bb4dbf None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:21.442854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c709d59-5e26-40a6-8532-fd4b92bb4dbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:21.443453 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c709d59-5e26-40a6-8532-fd4b92bb4dbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:21.446172 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c709d59-5e26-40a6-8532-fd4b92bb4dbf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:21.470405 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 489/964] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:21.480725 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c709d59-5e26-40a6-8532-fd4b92bb4dbf None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:21.495283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:21.520453 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 476/965] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:21 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:16:21.544842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa335c-2c34-42ff-a911-0bed3ad7b5a5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:21.641341 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c709d59-5e26-40a6-8532-fd4b92bb4dbf None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:21.682764 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2abe0-ec35-494e-859f-273806b859de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:21.685901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2abe0-ec35-494e-859f-273806b859de None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:21.685901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2abe0-ec35-494e-859f-273806b859de None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:21.685901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2abe0-ec35-494e-859f-273806b859de None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:21.692261 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2abe0-ec35-494e-859f-273806b859de None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:21.694558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2abe0-ec35-494e-859f-273806b859de None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:21.708208 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa335c-2c34-42ff-a911-0bed3ad7b5a5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:21.711642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-30e2abe0-ec35-494e-859f-273806b859de None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 18:16:21.711642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:21.711642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:21.711642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:21.711642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:21.711642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:21.711642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:21.711642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:21.711642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:21.711642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:21.711642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:21.711642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:21.711642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:16:21.711642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:16:21.711642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:16:21.711642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:16:21.712613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:21.712613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:21.712613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:16:21.712613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:16:21.712613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:16:21.712613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:16:21.712613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:21.712613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:21.712613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:21.712613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:21.712613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:21.712613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:21.712613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:21.712613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:21.712613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:21.712613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:21.712613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:21.712613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:21.713794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:21.713794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:21.713794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:16:21.713794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:16:21.713794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:16:21.713794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:16:21.713794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:16:21.713794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:16:21.713794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 18:16:21.713794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:21.713794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 490/966] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 30 18:16:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:21.714437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576228cb-db57-4b2f-a1a5-96083f70f6e1 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:21.714508 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576228cb-db57-4b2f-a1a5-96083f70f6e1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:21.715067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576228cb-db57-4b2f-a1a5-96083f70f6e1 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:21.720468 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 491/967] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:21.721854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd363071-e970-4732-b59b-11c4385a88d5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:21.722175 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd363071-e970-4732-b59b-11c4385a88d5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:21.722421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd363071-e970-4732-b59b-11c4385a88d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:21.743702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fd363071-e970-4732-b59b-11c4385a88d5 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:21.745636 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fd363071-e970-4732-b59b-11c4385a88d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:21.752790 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2c2e2-c4f9-4bad-995c-1b8357d15508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:21.753513 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2c2e2-c4f9-4bad-995c-1b8357d15508 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:21.753842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2c2e2-c4f9-4bad-995c-1b8357d15508 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:21.754116 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2c2e2-c4f9-4bad-995c-1b8357d15508 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:21.754829 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a2c2e2-c4f9-4bad-995c-1b8357d15508 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:21.755341 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:21.757385 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a2c2e2-c4f9-4bad-995c-1b8357d15508 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:21.758748 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: WARNING keystone.server.flask.common [None req-b1a2c2e2-c4f9-4bad-995c-1b8357d15508 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 30 18:16:21.773474 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.sql.core [None req-b1a2c2e2-c4f9-4bad-995c-1b8357d15508 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 18:16:21.773474 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 30 18:16:21.773474 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [parameters: {'user_id': 'ca9f05028201471c8ae50b9e5d868459', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 30 18:16:21.773474 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80206) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 18:16:21.781923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b1a2c2e2-c4f9-4bad-995c-1b8357d15508 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 30 18:16:21.781923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:21.781923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 18:16:21.781923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 18:16:21.781923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 18:16:21.781923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 18:16:21.781923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 18:16:21.781923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 18:16:21.781923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 18:16:21.781923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application conn.query(q) Jun 30 18:16:21.781923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 18:16:21.781923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 18:16:21.781923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 18:16:21.781923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result.read() Jun 30 18:16:21.781923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 18:16:21.783151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 18:16:21.783151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 18:16:21.783151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 18:16:21.783151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 18:16:21.783151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 18:16:21.783151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 18:16:21.783151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 18:16:21.783151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 18:16:21.783151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:21.783151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 18:16:21.783151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:21.783151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:21.783151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 18:16:21.783151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 18:16:21.783151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 30 18:16:21.783151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 30 18:16:21.783151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 18:16:21.783151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application next(self.gen) Jun 30 18:16:21.785506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 18:16:21.785506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application yield resource Jun 30 18:16:21.785506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 18:16:21.785506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application next(self.gen) Jun 30 18:16:21.785506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 18:16:21.785506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 18:16:21.785506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 18:16:21.785506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 18:16:21.785506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 18:16:21.785506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise self.value Jun 30 18:16:21.785506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 18:16:21.785506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 18:16:21.785506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 18:16:21.785506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application session.commit() Jun 30 18:16:21.785506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 18:16:21.785506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 18:16:21.785506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 18:16:21.786554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 18:16:21.786554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 18:16:21.786554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.session.flush() Jun 30 18:16:21.786554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 18:16:21.786554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 18:16:21.786554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 18:16:21.786554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 18:16:21.786554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 18:16:21.786554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application compat.raise_( Jun 30 18:16:21.786554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 18:16:21.786554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception Jun 30 18:16:21.786554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 18:16:21.786554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 18:16:21.786554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 18:16:21.786554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 18:16:21.786554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 18:16:21.786554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 18:16:21.789243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 18:16:21.789243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 18:16:21.789243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 30 18:16:21.789243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 30 18:16:21.789243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 18:16:21.789243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 18:16:21.789243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 18:16:21.789243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 18:16:21.789243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 18:16:21.789243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 18:16:21.789243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 18:16:21.789243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 18:16:21.789243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 18:16:21.789243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 18:16:21.789243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 18:16:21.789243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception Jun 30 18:16:21.793094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 18:16:21.793094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 18:16:21.793094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 18:16:21.793094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 18:16:21.793094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 18:16:21.793094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 18:16:21.793094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 18:16:21.793094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application conn.query(q) Jun 30 18:16:21.793094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 18:16:21.793094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 18:16:21.793094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 18:16:21.793094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application result.read() Jun 30 18:16:21.793094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 18:16:21.793094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 18:16:21.793094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 18:16:21.793094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 18:16:21.794500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 18:16:21.794500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 18:16:21.794500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 18:16:21.794500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 18:16:21.794500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 18:16:21.794500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 30 18:16:21.794500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ca9f05028201471c8ae50b9e5d868459', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 30 18:16:21.794500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 18:16:21.794500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:21.794500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:16:21.794500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:21.794500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:21.794500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:21.794500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:21.794500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:21.795490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:21.795490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:21.795490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:21.795490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:21.795490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:21.795490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:21.795490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:21.795490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 30 18:16:21.795490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 30 18:16:21.795490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:21.795490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:21.795490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:16:21.795490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:16:21.795490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:16:21.795490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:16:21.795490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 30 18:16:21.795490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 30 18:16:21.796964 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 30 18:16:21.796964 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 30 18:16:21.796964 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 18:16:21.796964 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 18:16:21.796964 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 30 18:16:21.796964 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:21.796964 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 477/968] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Jun 30 18:16:21 2022] POST /identity/v3/users => generated 160 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:16:21.796964 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2c2e2-c4f9-4bad-995c-1b8357d15508 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:21.841286 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 492/969] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:21 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:21.842185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd363071-e970-4732-b59b-11c4385a88d5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:21.858308 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:21.943666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2c2e2-c4f9-4bad-995c-1b8357d15508 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:21.957080 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd363071-e970-4732-b59b-11c4385a88d5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:21.996515 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f9b06f-50af-4a3d-acf4-2ad585210189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:21.997514 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9b06f-50af-4a3d-acf4-2ad585210189 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:22.000109 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9b06f-50af-4a3d-acf4-2ad585210189 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:22.000109 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9b06f-50af-4a3d-acf4-2ad585210189 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:22.010971 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f9b06f-50af-4a3d-acf4-2ad585210189 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:22.011206 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c3175-5aae-43bf-8308-2c40d7e54c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:22.012277 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c3175-5aae-43bf-8308-2c40d7e54c41 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:22.012792 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c3175-5aae-43bf-8308-2c40d7e54c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:22.013318 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c3175-5aae-43bf-8308-2c40d7e54c41 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:22.013422 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f9b06f-50af-4a3d-acf4-2ad585210189 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:22.015083 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c3175-5aae-43bf-8308-2c40d7e54c41 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:22.015652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c3175-5aae-43bf-8308-2c40d7e54c41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:22.018881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c3175-5aae-43bf-8308-2c40d7e54c41 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:22.038353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d8f9b06f-50af-4a3d-acf4-2ad585210189 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 18:16:22.038353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:22.038353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:22.038353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:22.038353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:22.038353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:22.038353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:22.038353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:22.038353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:22.038353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:22.038353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:22.038353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:22.038353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:16:22.038353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:16:22.038353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:16:22.038353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:16:22.040225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:22.040225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:22.040225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:16:22.040225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:16:22.040225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:16:22.040225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:16:22.040225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:22.040225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:22.040225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:22.040225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:22.040225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:22.040225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:22.040225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:22.040225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:22.040225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:22.040225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:22.040225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:22.040225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:22.042509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:22.042509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:22.042509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:16:22.042509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:16:22.042509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:16:22.042509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:16:22.042509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:16:22.042509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:16:22.042509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 18:16:22.042509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:22.044513 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 493/970] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:16:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:22.047087 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f9b06f-50af-4a3d-acf4-2ad585210189 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:22.050526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:22.060510 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 478/971] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Jun 30 18:16:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:22.060510 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c3175-5aae-43bf-8308-2c40d7e54c41 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:22.180060 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c3175-5aae-43bf-8308-2c40d7e54c41 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:22.190802 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:22.191008 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f9b06f-50af-4a3d-acf4-2ad585210189 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:22.230860 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195646e0-e344-4b29-9e3e-ac2fee816a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:22.231798 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe1b75-6d19-4e04-831f-9c6ab3b83aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:22.231992 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195646e0-e344-4b29-9e3e-ac2fee816a5f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:22.232215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195646e0-e344-4b29-9e3e-ac2fee816a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:22.232499 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195646e0-e344-4b29-9e3e-ac2fee816a5f None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:22.232634 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe1b75-6d19-4e04-831f-9c6ab3b83aff None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:22.232882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe1b75-6d19-4e04-831f-9c6ab3b83aff None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:22.233160 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe1b75-6d19-4e04-831f-9c6ab3b83aff None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:22.233555 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195646e0-e344-4b29-9e3e-ac2fee816a5f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:22.233858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe1b75-6d19-4e04-831f-9c6ab3b83aff None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:22.234120 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195646e0-e344-4b29-9e3e-ac2fee816a5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:22.235240 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe1b75-6d19-4e04-831f-9c6ab3b83aff None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:22.235572 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195646e0-e344-4b29-9e3e-ac2fee816a5f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:22.248549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d4fe1b75-6d19-4e04-831f-9c6ab3b83aff None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 18:16:22.248549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:22.248549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:16:22.248549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:16:22.248549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:16:22.248549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:16:22.248549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:16:22.248549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:22.248549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 18:16:22.248549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:22.248549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:16:22.248549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:22.248549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:22.248549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:22.248549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:22.248549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:22.248549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:22.250184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:22.250184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:22.250184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:22.250184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:22.250184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:22.250184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:22.250184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:16:22.250184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:16:22.250184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:16:22.250184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:16:22.250184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:22.250184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:22.250184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:22.250184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:22.250184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:22.250184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:22.251299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:22.251299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:22.251299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:22.251299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:22.251299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:22.251299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:22.251299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:22.251299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:22.251299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:22.251299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:22.251299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:16:22.251299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:22.251299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 18:16:22.251299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:22.251299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 494/972] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 30 18:16:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:22.252118 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe1b75-6d19-4e04-831f-9c6ab3b83aff None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:22.274379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 479/973] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 30 18:16:22 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:22.275830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93159a-7822-418c-b1ac-7bb4d2de7488 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:22.276121 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93159a-7822-418c-b1ac-7bb4d2de7488 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:22.276404 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93159a-7822-418c-b1ac-7bb4d2de7488 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:22.277743 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 480/974] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:22.278507 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93159a-7822-418c-b1ac-7bb4d2de7488 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:22.362168 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:22.409686 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe1b75-6d19-4e04-831f-9c6ab3b83aff None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:22.427063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93159a-7822-418c-b1ac-7bb4d2de7488 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:22.463812 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa856e2-272d-4dcf-bb99-d27b6ddd8f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:22.464630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa856e2-272d-4dcf-bb99-d27b6ddd8f6a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:22.465024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa856e2-272d-4dcf-bb99-d27b6ddd8f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:22.465416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa856e2-272d-4dcf-bb99-d27b6ddd8f6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:22.476785 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70bcba-6b8c-422d-83f9-7c17691e4b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:22.477465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70bcba-6b8c-422d-83f9-7c17691e4b49 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:22.477838 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70bcba-6b8c-422d-83f9-7c17691e4b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:22.478118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70bcba-6b8c-422d-83f9-7c17691e4b49 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:22.478904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70bcba-6b8c-422d-83f9-7c17691e4b49 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:22.479219 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70bcba-6b8c-422d-83f9-7c17691e4b49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:22.480624 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70bcba-6b8c-422d-83f9-7c17691e4b49 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:22.486874 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 481/975] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 30 18:16:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:22.488077 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b759e-3583-47ec-9d6d-e4b94fc7a309 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:22.488369 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b759e-3583-47ec-9d6d-e4b94fc7a309 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:22.488768 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b759e-3583-47ec-9d6d-e4b94fc7a309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:22.508682 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-eb0b759e-3583-47ec-9d6d-e4b94fc7a309 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:22.510295 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-eb0b759e-3583-47ec-9d6d-e4b94fc7a309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:22.544919 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:22.555163 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa856e2-272d-4dcf-bb99-d27b6ddd8f6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:22.556638 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa856e2-272d-4dcf-bb99-d27b6ddd8f6a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:22.598388 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 482/976] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:22 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:22.599446 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b759e-3583-47ec-9d6d-e4b94fc7a309 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:22.641853 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:22.682724 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 495/977] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:16:22 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 18:16:22.683599 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa856e2-272d-4dcf-bb99-d27b6ddd8f6a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:22.712278 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b759e-3583-47ec-9d6d-e4b94fc7a309 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:22.750754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a95da-7367-45fb-b38f-8484a52092be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:22.751307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a95da-7367-45fb-b38f-8484a52092be None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:22.751588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a95da-7367-45fb-b38f-8484a52092be None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:22.751865 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a95da-7367-45fb-b38f-8484a52092be None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:22.752575 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a95da-7367-45fb-b38f-8484a52092be None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:22.752883 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a95da-7367-45fb-b38f-8484a52092be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:22.754302 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a95da-7367-45fb-b38f-8484a52092be None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:22.760067 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 483/978] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 30 18:16:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:22.760881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a95da-7367-45fb-b38f-8484a52092be None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:22.786276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:22.805132 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa856e2-272d-4dcf-bb99-d27b6ddd8f6a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:22.841149 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37211720-d4a9-4ae8-a21d-568077d0fcb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:22.841995 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37211720-d4a9-4ae8-a21d-568077d0fcb5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:22.842268 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37211720-d4a9-4ae8-a21d-568077d0fcb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:22.842549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37211720-d4a9-4ae8-a21d-568077d0fcb5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:22.843205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37211720-d4a9-4ae8-a21d-568077d0fcb5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:22.844591 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37211720-d4a9-4ae8-a21d-568077d0fcb5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:22.856848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-37211720-d4a9-4ae8-a21d-568077d0fcb5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 18:16:22.856848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:22.856848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:16:22.856848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:16:22.856848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:16:22.856848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:16:22.856848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:16:22.856848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:22.856848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 18:16:22.856848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:22.856848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:16:22.856848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:22.856848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:22.856848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:22.856848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:22.856848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:22.856848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:22.858032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:22.858032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:22.858032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:22.858032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:22.858032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:22.858032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:22.858032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:16:22.858032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:16:22.858032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:16:22.858032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:16:22.858032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:22.858032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:22.858032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:22.858032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:22.858032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:22.858032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:22.859558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:22.859558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:22.859558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:22.859558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:22.859558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:22.859558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:22.859558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:22.859558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:22.859558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:22.859558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:22.859558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:16:22.859558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:22.859558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 18:16:22.859558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:22.859558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 496/979] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:16:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:22.861124 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37211720-d4a9-4ae8-a21d-568077d0fcb5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:22.876952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a95da-7367-45fb-b38f-8484a52092be None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:22.894659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:22.916904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d767e-f580-4af8-8e23-90f9bb68d43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:22.918200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d767e-f580-4af8-8e23-90f9bb68d43a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:22.918665 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d767e-f580-4af8-8e23-90f9bb68d43a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:22.919128 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d767e-f580-4af8-8e23-90f9bb68d43a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:22.936075 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d767e-f580-4af8-8e23-90f9bb68d43a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:22.938642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d767e-f580-4af8-8e23-90f9bb68d43a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:22.962718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-589d767e-f580-4af8-8e23-90f9bb68d43a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:22.962718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:22.962718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:22.962718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:22.962718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:22.962718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:22.962718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:22.962718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:22.962718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:22.962718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:22.962718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:22.962718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:22.962718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:16:22.962718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:16:22.962718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:16:22.963843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:16:22.963843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:22.963843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:22.963843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:22.963843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:22.963843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:22.963843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:22.963843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:22.963843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:22.963843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:22.963843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:22.963843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:22.963843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:22.963843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:22.963843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:22.963843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:22.963843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:22.965058 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:16:22.965058 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:16:22.965058 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:22.965058 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:22.965058 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 484/980] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:22.965058 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d767e-f580-4af8-8e23-90f9bb68d43a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:22.989317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37211720-d4a9-4ae8-a21d-568077d0fcb5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:23.029694 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd6cd6c-d8d1-4d45-a6bc-d08dd46d15be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:23.030369 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6cd6c-d8d1-4d45-a6bc-d08dd46d15be None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:23.030653 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6cd6c-d8d1-4d45-a6bc-d08dd46d15be None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:23.030937 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6cd6c-d8d1-4d45-a6bc-d08dd46d15be None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:23.031683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd6cd6c-d8d1-4d45-a6bc-d08dd46d15be None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:23.033101 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd6cd6c-d8d1-4d45-a6bc-d08dd46d15be None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:23.051142 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:23.060135 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 497/981] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 30 18:16:22 2022] POST /identity/v3/endpoints => generated 349 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:16:23.061200 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd6cd6c-d8d1-4d45-a6bc-d08dd46d15be None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:23.086216 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d767e-f580-4af8-8e23-90f9bb68d43a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:23.142645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e892c-4238-4137-96cf-5dda7bce574c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:23.142993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e892c-4238-4137-96cf-5dda7bce574c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:23.143317 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e892c-4238-4137-96cf-5dda7bce574c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:23.143558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e892c-4238-4137-96cf-5dda7bce574c None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:23.144286 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e892c-4238-4137-96cf-5dda7bce574c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:23.144607 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e892c-4238-4137-96cf-5dda7bce574c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:23.146118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e892c-4238-4137-96cf-5dda7bce574c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:23.152592 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 485/982] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 30 18:16:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:23.168575 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e892c-4238-4137-96cf-5dda7bce574c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:23.190457 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:23.219409 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd6cd6c-d8d1-4d45-a6bc-d08dd46d15be None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:23.278564 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08048340-dd4c-464d-8afd-51a1288dd29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:23.279140 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08048340-dd4c-464d-8afd-51a1288dd29d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:23.279428 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08048340-dd4c-464d-8afd-51a1288dd29d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:23.279704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08048340-dd4c-464d-8afd-51a1288dd29d None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:23.280444 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08048340-dd4c-464d-8afd-51a1288dd29d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:23.280756 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08048340-dd4c-464d-8afd-51a1288dd29d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:23.282314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08048340-dd4c-464d-8afd-51a1288dd29d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:23.298894 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 498/983] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:23.306336 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08048340-dd4c-464d-8afd-51a1288dd29d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:23.322150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e892c-4238-4137-96cf-5dda7bce574c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:23.367539 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74eb1a1-d931-4504-a106-b599cc490c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:23.368539 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74eb1a1-d931-4504-a106-b599cc490c9a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:23.368827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74eb1a1-d931-4504-a106-b599cc490c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:23.369111 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74eb1a1-d931-4504-a106-b599cc490c9a None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:23.370050 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74eb1a1-d931-4504-a106-b599cc490c9a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:23.370374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74eb1a1-d931-4504-a106-b599cc490c9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:23.372285 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74eb1a1-d931-4504-a106-b599cc490c9a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:23.374749 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:23.383777 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 486/984] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 30 18:16:23 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:23.385370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74eb1a1-d931-4504-a106-b599cc490c9a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:23.455671 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08048340-dd4c-464d-8afd-51a1288dd29d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:23.510294 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628056a-63bf-47d4-baa1-1b37c1dc4617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:23.511093 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628056a-63bf-47d4-baa1-1b37c1dc4617 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:23.511383 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628056a-63bf-47d4-baa1-1b37c1dc4617 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:23.511665 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628056a-63bf-47d4-baa1-1b37c1dc4617 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:23.516181 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74eb1a1-d931-4504-a106-b599cc490c9a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:23.518861 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628056a-63bf-47d4-baa1-1b37c1dc4617 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:23.520335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628056a-63bf-47d4-baa1-1b37c1dc4617 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:23.534897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-d628056a-63bf-47d4-baa1-1b37c1dc4617 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:16:23.534897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:23.534897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:23.534897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:23.534897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:23.534897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:23.534897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:23.534897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:23.534897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:23.534897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:23.534897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:23.534897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:23.534897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:16:23.534897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:16:23.534897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:16:23.536604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:16:23.536604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:23.536604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:23.536604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:23.536604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:23.536604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:23.536604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:23.536604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:23.536604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:23.536604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:23.536604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:23.536604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:23.536604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:23.536604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:23.536604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:23.536604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:23.536604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:23.538801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:16:23.538801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:16:23.538801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:16:23.538801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:16:23.538801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:16:23.538801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:16:23.538801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:16:23.538801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:23.541524 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 499/985] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:16:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:16:23.542486 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628056a-63bf-47d4-baa1-1b37c1dc4617 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:23.552033 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:23.561968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ccd4c4-2566-4e91-b34c-435357d4639f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:23.562732 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccd4c4-2566-4e91-b34c-435357d4639f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:23.563019 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccd4c4-2566-4e91-b34c-435357d4639f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:23.563321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccd4c4-2566-4e91-b34c-435357d4639f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:23.564188 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ccd4c4-2566-4e91-b34c-435357d4639f None admin] RBAC: Adding query filter params (interface=public, service_id=1c45961d4ee5432fabce95ed26606c00, region_id=RegionOne) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:23.564500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ccd4c4-2566-4e91-b34c-435357d4639f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:23.567195 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ccd4c4-2566-4e91-b34c-435357d4639f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:23.576497 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 487/986] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 30 18:16:23 2022] GET /identity/v3/endpoints?service_id=1c45961d4ee5432fabce95ed26606c00&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:23.577885 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ccd4c4-2566-4e91-b34c-435357d4639f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:23.661016 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:23.676987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628056a-63bf-47d4-baa1-1b37c1dc4617 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:23.708257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ccd4c4-2566-4e91-b34c-435357d4639f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:23.718498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa79bf-8aa3-46c2-880e-fc26f8ade3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:23.719204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa79bf-8aa3-46c2-880e-fc26f8ade3cf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:23.719631 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa79bf-8aa3-46c2-880e-fc26f8ade3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:23.720309 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa79bf-8aa3-46c2-880e-fc26f8ade3cf None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:23.721190 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa79bf-8aa3-46c2-880e-fc26f8ade3cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:23.721850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa79bf-8aa3-46c2-880e-fc26f8ade3cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:23.723485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa79bf-8aa3-46c2-880e-fc26f8ade3cf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:23.737765 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 500/987] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:16:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:23.744948 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa79bf-8aa3-46c2-880e-fc26f8ade3cf None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:23.753887 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734ab94-3acc-407c-8686-62f60ea63198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:23.754623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734ab94-3acc-407c-8686-62f60ea63198 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:23.755068 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734ab94-3acc-407c-8686-62f60ea63198 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:23.755402 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734ab94-3acc-407c-8686-62f60ea63198 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:23.756747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734ab94-3acc-407c-8686-62f60ea63198 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:23.758237 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734ab94-3acc-407c-8686-62f60ea63198 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:23.764780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 488/988] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 30 18:16:23 2022] GET /identity/v3/services => generated 1727 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:16:23.832453 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:23.873371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa79bf-8aa3-46c2-880e-fc26f8ade3cf None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:23.910763 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b1524-ffaf-47bf-8bf9-d77367ead305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:23.911528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b1524-ffaf-47bf-8bf9-d77367ead305 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:23.911979 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b1524-ffaf-47bf-8bf9-d77367ead305 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:23.912479 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b1524-ffaf-47bf-8bf9-d77367ead305 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:23.913411 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6b1524-ffaf-47bf-8bf9-d77367ead305 None admin] RBAC: Adding query filter params (role.id=fca7293e63b14558bb206aaac4e5d1ca, scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b, user.id=4ddcf6306e704fe9ad5ed8779178ae67) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:23.913836 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6b1524-ffaf-47bf-8bf9-d77367ead305 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:23.915385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6b1524-ffaf-47bf-8bf9-d77367ead305 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:23.922604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 501/989] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:16:23 2022] GET /identity/v3/role_assignments?user.id=4ddcf6306e704fe9ad5ed8779178ae67&scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b&role.id=fca7293e63b14558bb206aaac4e5d1ca => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:25.183027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6675da6-cd6b-4ef1-8182-1228e80a0490 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:25.183543 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6675da6-cd6b-4ef1-8182-1228e80a0490 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:25.183629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6675da6-cd6b-4ef1-8182-1228e80a0490 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:25.184992 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 489/990] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:25.190264 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a05d7-6674-4032-a699-fa8f8d211efb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:25.190665 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a05d7-6674-4032-a699-fa8f8d211efb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:25.190828 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a05d7-6674-4032-a699-fa8f8d211efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:25.207467 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-582a05d7-6674-4032-a699-fa8f8d211efb None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:25.208658 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-582a05d7-6674-4032-a699-fa8f8d211efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:25.277908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 502/991] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:25 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:25.300397 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6675da6-cd6b-4ef1-8182-1228e80a0490 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:25.410531 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6675da6-cd6b-4ef1-8182-1228e80a0490 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:25.439167 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:25.448731 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407e274-a283-43dc-9826-0be67cea5b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:25.449539 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407e274-a283-43dc-9826-0be67cea5b56 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:25.449852 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407e274-a283-43dc-9826-0be67cea5b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:25.450129 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407e274-a283-43dc-9826-0be67cea5b56 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:25.450788 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407e274-a283-43dc-9826-0be67cea5b56 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:25.452175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407e274-a283-43dc-9826-0be67cea5b56 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:25.464816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b407e274-a283-43dc-9826-0be67cea5b56 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 18:16:25.464816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:25.464816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:16:25.464816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:16:25.464816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:16:25.464816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:16:25.464816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:16:25.464816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:25.464816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 18:16:25.464816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:25.464816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:16:25.464816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:25.464816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:25.464816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:25.464816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:25.464816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:25.464816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:25.465905 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:25.465905 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:25.465905 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:25.465905 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:25.465905 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:25.465905 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:25.465905 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:16:25.465905 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:16:25.465905 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:16:25.465905 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:16:25.465905 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:25.465905 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:25.465905 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:25.465905 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:25.465905 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:25.465905 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:25.466738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:25.466738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:25.466738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:25.466738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:25.466738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:25.466738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:25.466738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:25.466738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:25.466738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:25.466738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:25.466738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:16:25.466738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:25.466738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 18:16:25.466738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:25.468260 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 490/992] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 30 18:16:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:25.472220 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a05d7-6674-4032-a699-fa8f8d211efb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:25.532058 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:25.576798 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a05d7-6674-4032-a699-fa8f8d211efb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:25.612464 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215eb56-7896-4509-b552-5b49b12f0e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:25.613127 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215eb56-7896-4509-b552-5b49b12f0e76 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:25.613414 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215eb56-7896-4509-b552-5b49b12f0e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:25.613729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215eb56-7896-4509-b552-5b49b12f0e76 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:25.614438 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215eb56-7896-4509-b552-5b49b12f0e76 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:25.614755 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215eb56-7896-4509-b552-5b49b12f0e76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:25.616365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215eb56-7896-4509-b552-5b49b12f0e76 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:25.622549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 503/993] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 30 18:16:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:25.627245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407e274-a283-43dc-9826-0be67cea5b56 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:25.733124 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407e274-a283-43dc-9826-0be67cea5b56 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:25.771232 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4e3b2-2dac-4f51-96a7-674bd3dde866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:25.771897 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4e3b2-2dac-4f51-96a7-674bd3dde866 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:25.772181 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4e3b2-2dac-4f51-96a7-674bd3dde866 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:25.772458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4e3b2-2dac-4f51-96a7-674bd3dde866 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:25.773173 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee4e3b2-2dac-4f51-96a7-674bd3dde866 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:25.773477 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee4e3b2-2dac-4f51-96a7-674bd3dde866 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:25.774890 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee4e3b2-2dac-4f51-96a7-674bd3dde866 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:25.785398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 491/994] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 30 18:16:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:25.787917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215eb56-7896-4509-b552-5b49b12f0e76 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:25.906956 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc85188-4f6d-4d34-a21c-ebe4122bac1d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:25.907255 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc85188-4f6d-4d34-a21c-ebe4122bac1d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:25.907629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc85188-4f6d-4d34-a21c-ebe4122bac1d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:25.908895 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 492/995] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:16:25.913245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215eb56-7896-4509-b552-5b49b12f0e76 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:25.915880 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0a0dd-fe7d-461a-9554-03a217244b24 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:25.916199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0a0dd-fe7d-461a-9554-03a217244b24 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:25.916502 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0a0dd-fe7d-461a-9554-03a217244b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:25.940213 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f7d0a0dd-fe7d-461a-9554-03a217244b24 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:25.942580 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f7d0a0dd-fe7d-461a-9554-03a217244b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:25.947427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:25.960896 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd4f2f-3488-4270-aded-cbd9c83ad306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:25.961596 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd4f2f-3488-4270-aded-cbd9c83ad306 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:25.961920 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd4f2f-3488-4270-aded-cbd9c83ad306 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:25.962377 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd4f2f-3488-4270-aded-cbd9c83ad306 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:25.962991 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd4f2f-3488-4270-aded-cbd9c83ad306 None admin] RBAC: Adding query filter params (interface=admin, service_id=723ffd33dc5e4957b0564e112f9558a8, region_id=RegionOne) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:25.963402 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd4f2f-3488-4270-aded-cbd9c83ad306 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:25.964704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd4f2f-3488-4270-aded-cbd9c83ad306 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:25.971397 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 504/996] 10.222.0.33 () {56 vars in 1249 bytes} [Thu Jun 30 18:16:25 2022] GET /identity/v3/endpoints?service_id=723ffd33dc5e4957b0564e112f9558a8&interface=admin®ion_id=RegionOne => generated 193 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:25.978179 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd4f2f-3488-4270-aded-cbd9c83ad306 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:26.018764 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:26.039998 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 493/997] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:25 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:16:26.053849 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d31c7-7cce-4ea9-ae54-11c2f4bcf1a9 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:26.054139 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d31c7-7cce-4ea9-ae54-11c2f4bcf1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:26.054424 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d31c7-7cce-4ea9-ae54-11c2f4bcf1a9 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:26.055812 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 494/998] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:16:26.061013 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8aca8-a99c-4a40-a21d-496452d427a4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:26.061301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8aca8-a99c-4a40-a21d-496452d427a4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:26.061611 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8aca8-a99c-4a40-a21d-496452d427a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:26.080968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-35d8aca8-a99c-4a40-a21d-496452d427a4 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:26.082214 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-35d8aca8-a99c-4a40-a21d-496452d427a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:26.103472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd4f2f-3488-4270-aded-cbd9c83ad306 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:26.139548 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf35e1-5089-450e-9e63-9fee65d0ff78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:26.140216 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf35e1-5089-450e-9e63-9fee65d0ff78 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:26.140520 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf35e1-5089-450e-9e63-9fee65d0ff78 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:26.140761 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf35e1-5089-450e-9e63-9fee65d0ff78 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:26.141439 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf35e1-5089-450e-9e63-9fee65d0ff78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:26.143004 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf35e1-5089-450e-9e63-9fee65d0ff78 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:26.150975 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 505/999] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 30 18:16:25 2022] GET /identity/v3/services => generated 1727 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:16:26.151881 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf35e1-5089-450e-9e63-9fee65d0ff78 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:26.155984 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 495/1000] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:26 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:16:26.182870 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8aca8-a99c-4a40-a21d-496452d427a4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:26.243254 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:26.309418 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf35e1-5089-450e-9e63-9fee65d0ff78 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:26.333540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8aca8-a99c-4a40-a21d-496452d427a4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:26.356459 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90e7d2-a579-4dda-ad07-0b97e102d089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:26.357286 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90e7d2-a579-4dda-ad07-0b97e102d089 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:26.357657 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90e7d2-a579-4dda-ad07-0b97e102d089 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:26.357909 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90e7d2-a579-4dda-ad07-0b97e102d089 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:26.358636 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90e7d2-a579-4dda-ad07-0b97e102d089 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:26.359963 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90e7d2-a579-4dda-ad07-0b97e102d089 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:26.363640 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:26.372598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-4c90e7d2-a579-4dda-ad07-0b97e102d089 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 18:16:26.372598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:26.372598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:16:26.372598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:16:26.372598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:16:26.372598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:16:26.372598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:16:26.372598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:26.372598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 18:16:26.372598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:26.372598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:16:26.372598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:26.372598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:26.372598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:26.372598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:26.372598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:26.372598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:26.373830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:26.373830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:26.373830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:26.373830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:26.373830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:26.373830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:26.373830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:16:26.373830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:16:26.373830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:16:26.373830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:16:26.373830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:26.373830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:26.373830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:26.373830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:26.373830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:26.373830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:26.379607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:26.379607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:26.379607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:26.379607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:26.379607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:26.379607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:26.379607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:26.379607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:26.379607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:26.379607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:26.379607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:16:26.379607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:26.379607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 18:16:26.379607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:26.379607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 506/1001] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:16:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:26.381501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90e7d2-a579-4dda-ad07-0b97e102d089 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:26.384834 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f602ca1-3258-4125-9d5b-bb29cf549842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:26.385644 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f602ca1-3258-4125-9d5b-bb29cf549842 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:26.385933 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f602ca1-3258-4125-9d5b-bb29cf549842 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:26.386210 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f602ca1-3258-4125-9d5b-bb29cf549842 None admin] PATH_INFO: `/v3/services/image` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:26.386863 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f602ca1-3258-4125-9d5b-bb29cf549842 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:26.388243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f602ca1-3258-4125-9d5b-bb29cf549842 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:26.400751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-0f602ca1-3258-4125-9d5b-bb29cf549842 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 18:16:26.400751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:26.400751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:16:26.400751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:16:26.400751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:16:26.400751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:16:26.400751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:16:26.400751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:26.400751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 18:16:26.400751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:26.400751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:16:26.400751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:26.400751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:26.400751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:26.400751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:26.400751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:26.400751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:26.402234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:26.402234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:26.402234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:26.402234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:26.402234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:26.402234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:26.402234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:16:26.402234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:16:26.402234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:16:26.402234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:16:26.402234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:26.402234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:26.402234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:26.402234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:26.402234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:26.402234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:26.403175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:26.403175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:26.403175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:26.403175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:26.403175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:26.403175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:26.403175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:26.403175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:26.403175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:26.403175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:26.403175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:16:26.403175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:26.403175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 18:16:26.403175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:26.407051 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 496/1002] 10.222.0.33 () {56 vars in 1105 bytes} [Thu Jun 30 18:16:26 2022] GET /identity/v3/services/image => generated 86 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:16:26.407940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f602ca1-3258-4125-9d5b-bb29cf549842 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:26.493602 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90e7d2-a579-4dda-ad07-0b97e102d089 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:26.510762 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:26.516199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f602ca1-3258-4125-9d5b-bb29cf549842 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:26.529983 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49dfde-10b7-4846-b144-3011873896e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:26.530854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49dfde-10b7-4846-b144-3011873896e1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:26.530996 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49dfde-10b7-4846-b144-3011873896e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:26.531273 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49dfde-10b7-4846-b144-3011873896e1 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:26.532030 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49dfde-10b7-4846-b144-3011873896e1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:26.532376 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49dfde-10b7-4846-b144-3011873896e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:26.533921 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49dfde-10b7-4846-b144-3011873896e1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:26.540853 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 507/1003] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 30 18:16:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:26.546829 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49dfde-10b7-4846-b144-3011873896e1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:26.556531 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c3938-9adf-4ed3-bb22-7f1b0935ffcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:26.557200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c3938-9adf-4ed3-bb22-7f1b0935ffcc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:26.557493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c3938-9adf-4ed3-bb22-7f1b0935ffcc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:26.557808 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c3938-9adf-4ed3-bb22-7f1b0935ffcc None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:26.558527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c3938-9adf-4ed3-bb22-7f1b0935ffcc None admin] RBAC: Adding query filter params (name=image) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:26.558842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c3938-9adf-4ed3-bb22-7f1b0935ffcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:26.560279 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c3938-9adf-4ed3-bb22-7f1b0935ffcc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:26.565623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 497/1004] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:26.571190 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c3938-9adf-4ed3-bb22-7f1b0935ffcc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:26.622520 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:26.661267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49dfde-10b7-4846-b144-3011873896e1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:26.692947 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c3938-9adf-4ed3-bb22-7f1b0935ffcc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:26.706089 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c1633-56aa-4e75-9ac0-b6dc0abb7ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:26.706764 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c1633-56aa-4e75-9ac0-b6dc0abb7ade None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:26.707055 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c1633-56aa-4e75-9ac0-b6dc0abb7ade None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:26.707343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c1633-56aa-4e75-9ac0-b6dc0abb7ade None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:26.708374 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c1633-56aa-4e75-9ac0-b6dc0abb7ade None admin] RBAC: Adding query filter params (type=compute) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:26.708374 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c1633-56aa-4e75-9ac0-b6dc0abb7ade None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:26.710038 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c1633-56aa-4e75-9ac0-b6dc0abb7ade None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:26.720053 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 508/1005] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 30 18:16:26 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:26.721300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c1633-56aa-4e75-9ac0-b6dc0abb7ade None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:26.730624 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c03f83-f63e-41fa-b812-daac7dbbeae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:26.731345 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c03f83-f63e-41fa-b812-daac7dbbeae3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:26.731636 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c03f83-f63e-41fa-b812-daac7dbbeae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:26.731920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c03f83-f63e-41fa-b812-daac7dbbeae3 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:26.732637 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c03f83-f63e-41fa-b812-daac7dbbeae3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:26.732950 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c03f83-f63e-41fa-b812-daac7dbbeae3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:26.734429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c03f83-f63e-41fa-b812-daac7dbbeae3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:26.739040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 498/1006] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:26 2022] GET /identity/v3/services?type=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:26.784493 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:26.846989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c1633-56aa-4e75-9ac0-b6dc0abb7ade None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:26.884469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1c999-0177-41f7-a3ab-7d3c6f541765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:26.885451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1c999-0177-41f7-a3ab-7d3c6f541765 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:26.885962 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1c999-0177-41f7-a3ab-7d3c6f541765 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:26.886457 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1c999-0177-41f7-a3ab-7d3c6f541765 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:26.887385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1c999-0177-41f7-a3ab-7d3c6f541765 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:26.888933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1c999-0177-41f7-a3ab-7d3c6f541765 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:26.910986 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 509/1007] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 30 18:16:26 2022] POST /identity/v3/endpoints => generated 334 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:16:28.335590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b7234-4c29-419c-adfe-ac5621946f53 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:28.336129 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b7234-4c29-419c-adfe-ac5621946f53 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:28.336129 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b7234-4c29-419c-adfe-ac5621946f53 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:28.337645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 499/1008] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:28.342971 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869b246-de10-401f-b24c-b5edd4083678 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:28.343430 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869b246-de10-401f-b24c-b5edd4083678 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:28.343517 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869b246-de10-401f-b24c-b5edd4083678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:28.361230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d869b246-de10-401f-b24c-b5edd4083678 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:28.362471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d869b246-de10-401f-b24c-b5edd4083678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:28.442449 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 510/1009] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:28 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:28.467842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b7234-4c29-419c-adfe-ac5621946f53 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:28.599187 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b7234-4c29-419c-adfe-ac5621946f53 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:28.610370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:28.652327 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0ccfa-250e-47cb-b2ed-d3c40ddaaf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:28.653675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0ccfa-250e-47cb-b2ed-d3c40ddaaf56 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:28.654167 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0ccfa-250e-47cb-b2ed-d3c40ddaaf56 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:28.654642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0ccfa-250e-47cb-b2ed-d3c40ddaaf56 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:28.655859 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc0ccfa-250e-47cb-b2ed-d3c40ddaaf56 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:28.658688 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc0ccfa-250e-47cb-b2ed-d3c40ddaaf56 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:28.681349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-adc0ccfa-250e-47cb-b2ed-d3c40ddaaf56 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 18:16:28.681349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:28.681349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:16:28.681349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:16:28.681349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:16:28.681349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:16:28.681349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:16:28.681349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:28.681349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 18:16:28.681349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:28.681349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:16:28.681349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:28.681349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:28.681349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:28.681349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:28.681349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:28.681349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:28.682630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:28.682630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:28.682630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:28.682630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:28.682630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:28.682630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:28.682630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:16:28.682630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:16:28.682630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:16:28.682630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:16:28.682630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:28.682630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:28.682630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:28.682630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:28.682630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:28.682630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:28.683550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:28.683550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:28.683550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:28.683550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:28.683550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:28.683550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:28.683550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:28.683550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:28.683550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:28.683550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:28.683550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:16:28.683550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:28.683550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 18:16:28.683550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:28.684533 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 500/1010] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 30 18:16:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:28.694003 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869b246-de10-401f-b24c-b5edd4083678 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:28.729868 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:28.820203 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869b246-de10-401f-b24c-b5edd4083678 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:28.844398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8a1ef-c811-4bc2-a5be-544fe21bab9d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:28.844777 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8a1ef-c811-4bc2-a5be-544fe21bab9d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:28.845092 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8a1ef-c811-4bc2-a5be-544fe21bab9d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:28.847130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 501/1011] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:28.853006 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876ae8a-fd56-4c40-8ade-5fddbe23eb3d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:28.853234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876ae8a-fd56-4c40-8ade-5fddbe23eb3d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:28.853485 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876ae8a-fd56-4c40-8ade-5fddbe23eb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:28.862902 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394b0ca-9526-4fa2-bcfd-c93eac6786b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:28.863565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394b0ca-9526-4fa2-bcfd-c93eac6786b9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:28.863848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394b0ca-9526-4fa2-bcfd-c93eac6786b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:28.864120 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394b0ca-9526-4fa2-bcfd-c93eac6786b9 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:28.864826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394b0ca-9526-4fa2-bcfd-c93eac6786b9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:28.865688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394b0ca-9526-4fa2-bcfd-c93eac6786b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:28.867067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394b0ca-9526-4fa2-bcfd-c93eac6786b9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:28.872486 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 511/1012] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 30 18:16:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:28.872752 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b876ae8a-fd56-4c40-8ade-5fddbe23eb3d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:28.873947 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b876ae8a-fd56-4c40-8ade-5fddbe23eb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:28.878035 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394b0ca-9526-4fa2-bcfd-c93eac6786b9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:28.957941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 502/1013] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:28 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:16:28.976918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876ae8a-fd56-4c40-8ade-5fddbe23eb3d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:28.989134 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:29.001766 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394b0ca-9526-4fa2-bcfd-c93eac6786b9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:29.043168 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5bddf-baed-409e-a61e-075ba8818067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:29.043838 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5bddf-baed-409e-a61e-075ba8818067 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:29.044124 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5bddf-baed-409e-a61e-075ba8818067 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:29.044435 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5bddf-baed-409e-a61e-075ba8818067 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:29.045146 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5bddf-baed-409e-a61e-075ba8818067 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:29.045454 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5bddf-baed-409e-a61e-075ba8818067 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:29.047206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5bddf-baed-409e-a61e-075ba8818067 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:29.057976 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 512/1014] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 30 18:16:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:29.058906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5bddf-baed-409e-a61e-075ba8818067 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:29.095975 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876ae8a-fd56-4c40-8ade-5fddbe23eb3d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:29.097940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:29.145139 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6715a-da4a-4798-970a-18079a959d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:29.145829 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6715a-da4a-4798-970a-18079a959d53 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:29.146123 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6715a-da4a-4798-970a-18079a959d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:29.146960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6715a-da4a-4798-970a-18079a959d53 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:29.148337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a6715a-da4a-4798-970a-18079a959d53 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:29.149758 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a6715a-da4a-4798-970a-18079a959d53 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:29.165480 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 503/1015] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 30 18:16:28 2022] POST /identity/v3/services => generated 242 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:16:29.166839 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817ebbb-2241-4fbf-8b3e-bad56c10d86f None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:29.167136 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817ebbb-2241-4fbf-8b3e-bad56c10d86f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:29.167420 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817ebbb-2241-4fbf-8b3e-bad56c10d86f None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:29.168670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 504/1016] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:29.174762 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cae541-bc80-43d8-a55c-0b9890a65a6d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:29.175101 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cae541-bc80-43d8-a55c-0b9890a65a6d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:29.175330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cae541-bc80-43d8-a55c-0b9890a65a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:29.189664 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5bddf-baed-409e-a61e-075ba8818067 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:29.192671 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f6cae541-bc80-43d8-a55c-0b9890a65a6d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:29.195022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f6cae541-bc80-43d8-a55c-0b9890a65a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:29.236105 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faecb09b-2ad1-428a-bf8c-49082e7412b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:29.236776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faecb09b-2ad1-428a-bf8c-49082e7412b3 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:29.237064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faecb09b-2ad1-428a-bf8c-49082e7412b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:29.237339 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faecb09b-2ad1-428a-bf8c-49082e7412b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:29.238191 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faecb09b-2ad1-428a-bf8c-49082e7412b3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:29.239597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faecb09b-2ad1-428a-bf8c-49082e7412b3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:29.266042 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 513/1017] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 30 18:16:29 2022] POST /identity/v3/endpoints => generated 325 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:16:29.291639 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 505/1018] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:16:29.319196 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faecb09b-2ad1-428a-bf8c-49082e7412b3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:29.337603 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:29.457744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faecb09b-2ad1-428a-bf8c-49082e7412b3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:29.494090 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859b93e-7659-4345-a35a-a5625bb8f4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:29.494913 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859b93e-7659-4345-a35a-a5625bb8f4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:29.495204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859b93e-7659-4345-a35a-a5625bb8f4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:29.495478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859b93e-7659-4345-a35a-a5625bb8f4e1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:29.506172 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0859b93e-7659-4345-a35a-a5625bb8f4e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:16:29.514334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-0859b93e-7659-4345-a35a-a5625bb8f4e1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 18:16:29.514334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:29.514334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:29.514334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:29.514334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:29.514334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:29.514334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:29.514334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:29.514334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:29.514334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:29.514334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:29.514334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:29.514334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:16:29.514334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:16:29.514334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:16:29.514334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:16:29.515265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:16:29.515265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:16:29.515265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:16:29.515265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:16:29.515265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:29.515265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:29.515265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:29.515265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:29.515265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:29.515265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:29.515265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:29.515265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:29.515265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:29.515265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:29.515265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:29.515265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:29.516144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:29.516144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:29.516144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:29.516144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:29.516144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:16:29.516144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:16:29.516144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:16:29.516144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:16:29.516144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:16:29.516144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:16:29.516144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 18:16:29.516144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:29.516144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 514/1019] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Jun 30 18:16:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:29.521562 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cae541-bc80-43d8-a55c-0b9890a65a6d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:29.614221 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:29.631755 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cae541-bc80-43d8-a55c-0b9890a65a6d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:29.669395 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79903dd-25bb-4add-b1e8-a8d371aeb610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:29.670093 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79903dd-25bb-4add-b1e8-a8d371aeb610 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:29.670375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79903dd-25bb-4add-b1e8-a8d371aeb610 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:29.670650 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79903dd-25bb-4add-b1e8-a8d371aeb610 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:29.671652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79903dd-25bb-4add-b1e8-a8d371aeb610 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:29.671959 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79903dd-25bb-4add-b1e8-a8d371aeb610 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:29.673327 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79903dd-25bb-4add-b1e8-a8d371aeb610 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:29.686697 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 506/1020] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 30 18:16:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:29.688324 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859b93e-7659-4345-a35a-a5625bb8f4e1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:29.789205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:29.824339 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859b93e-7659-4345-a35a-a5625bb8f4e1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:29.861607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0b2e3-1efc-4b4b-865e-e8823e18c009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:29.862441 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0b2e3-1efc-4b4b-865e-e8823e18c009 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:29.862805 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0b2e3-1efc-4b4b-865e-e8823e18c009 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:29.863007 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0b2e3-1efc-4b4b-865e-e8823e18c009 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:29.877599 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0b2e3-1efc-4b4b-865e-e8823e18c009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:29.879305 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0b2e3-1efc-4b4b-865e-e8823e18c009 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:29.899262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5de0b2e3-1efc-4b4b-865e-e8823e18c009 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:29.899262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:29.899262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:29.899262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:29.899262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:29.899262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:29.899262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:29.899262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:29.899262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:29.899262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:29.899262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:29.899262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:29.899262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:16:29.899262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:16:29.899262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:16:29.900465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:16:29.900465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:29.900465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:29.900465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:29.900465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:29.900465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:29.900465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:29.900465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:29.900465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:29.900465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:29.900465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:29.900465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:29.900465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:29.900465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:29.900465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:29.900465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:29.900465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:29.902974 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:16:29.902974 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:16:29.902974 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:29.902974 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:29.902974 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 515/1021] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:29.906449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79903dd-25bb-4add-b1e8-a8d371aeb610 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:30.040067 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79903dd-25bb-4add-b1e8-a8d371aeb610 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:30.106654 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906b0e2-856a-45b9-8852-6238f21b598d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:30.107851 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906b0e2-856a-45b9-8852-6238f21b598d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:30.108233 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906b0e2-856a-45b9-8852-6238f21b598d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:30.108582 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906b0e2-856a-45b9-8852-6238f21b598d None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:30.109410 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6906b0e2-856a-45b9-8852-6238f21b598d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:30.109815 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6906b0e2-856a-45b9-8852-6238f21b598d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:30.112421 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6906b0e2-856a-45b9-8852-6238f21b598d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:30.150882 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 507/1022] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:30.152881 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0b2e3-1efc-4b4b-865e-e8823e18c009 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:30.269825 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0b2e3-1efc-4b4b-865e-e8823e18c009 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:30.294248 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:30.307652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04648a1-b3d1-4c67-97c2-3277e2a4f0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:30.308442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04648a1-b3d1-4c67-97c2-3277e2a4f0b7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:30.308724 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04648a1-b3d1-4c67-97c2-3277e2a4f0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:30.308999 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04648a1-b3d1-4c67-97c2-3277e2a4f0b7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:30.317158 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04648a1-b3d1-4c67-97c2-3277e2a4f0b7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:30.318579 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04648a1-b3d1-4c67-97c2-3277e2a4f0b7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:30.335026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-c04648a1-b3d1-4c67-97c2-3277e2a4f0b7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 18:16:30.335026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:30.335026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:30.335026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:30.335026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:30.335026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:30.335026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:30.335026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:30.335026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:30.335026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:30.335026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:30.335026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:30.335026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:16:30.335026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:16:30.335026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:16:30.335026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:16:30.336222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:30.336222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:30.336222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:16:30.336222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:16:30.336222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:16:30.336222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:16:30.336222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:30.336222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:30.336222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:30.336222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:30.336222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:30.336222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:30.336222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:30.336222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:30.336222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:30.336222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:30.336222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:30.336222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:30.337245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:30.337245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:30.337245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:16:30.337245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:16:30.337245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:16:30.337245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:16:30.337245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:16:30.337245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:16:30.337245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 18:16:30.337245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:30.337245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 516/1023] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 30 18:16:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:30.342192 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906b0e2-856a-45b9-8852-6238f21b598d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:30.355707 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:30.467311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906b0e2-856a-45b9-8852-6238f21b598d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:30.504159 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713381c-98c7-4914-997a-372ac10acfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:30.504851 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713381c-98c7-4914-997a-372ac10acfcc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:30.505132 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713381c-98c7-4914-997a-372ac10acfcc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:30.505408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713381c-98c7-4914-997a-372ac10acfcc None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:30.507786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713381c-98c7-4914-997a-372ac10acfcc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:30.508103 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713381c-98c7-4914-997a-372ac10acfcc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:30.509649 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713381c-98c7-4914-997a-372ac10acfcc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:30.533891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 508/1024] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 30 18:16:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:30.539062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04648a1-b3d1-4c67-97c2-3277e2a4f0b7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:30.647617 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04648a1-b3d1-4c67-97c2-3277e2a4f0b7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:30.683133 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a9ed1-f358-4648-a1e2-77bd55bcd3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:30.683780 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a9ed1-f358-4648-a1e2-77bd55bcd3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:30.684071 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a9ed1-f358-4648-a1e2-77bd55bcd3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:30.684355 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a9ed1-f358-4648-a1e2-77bd55bcd3f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:30.733989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:30.753436 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a9ed1-f358-4648-a1e2-77bd55bcd3f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:30.754946 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a9ed1-f358-4648-a1e2-77bd55bcd3f1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:30.860681 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 517/1025] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:16:30 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 18:16:30.862043 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713381c-98c7-4914-997a-372ac10acfcc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:30.930943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:30.966829 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713381c-98c7-4914-997a-372ac10acfcc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:31.002194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec40c6f-88af-4727-aa0f-64fdb2f2f192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:31.002980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec40c6f-88af-4727-aa0f-64fdb2f2f192 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:31.003262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec40c6f-88af-4727-aa0f-64fdb2f2f192 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:31.003538 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec40c6f-88af-4727-aa0f-64fdb2f2f192 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:31.013842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec40c6f-88af-4727-aa0f-64fdb2f2f192 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:31.015297 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec40c6f-88af-4727-aa0f-64fdb2f2f192 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:31.035934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-aec40c6f-88af-4727-aa0f-64fdb2f2f192 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:31.035934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:31.035934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:31.035934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:31.035934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:31.035934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:31.035934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:31.035934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:31.035934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:31.035934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:31.035934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:31.035934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:31.035934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:16:31.035934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:16:31.035934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:16:31.037436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:16:31.037436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:31.037436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:31.037436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:31.037436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:31.037436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:31.037436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:31.037436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:31.037436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:31.037436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:31.037436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:31.037436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:31.037436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:31.037436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:31.037436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:31.037436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:31.037436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:31.043592 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:16:31.043592 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:16:31.043592 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:31.043592 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:31.043592 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 509/1026] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:31.043882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a9ed1-f358-4648-a1e2-77bd55bcd3f1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:31.140409 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:31.164735 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a9ed1-f358-4648-a1e2-77bd55bcd3f1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:31.207817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd72832-d653-467b-abb8-094645827006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:31.208496 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd72832-d653-467b-abb8-094645827006 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:31.209325 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd72832-d653-467b-abb8-094645827006 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:31.209886 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd72832-d653-467b-abb8-094645827006 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:31.210781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd72832-d653-467b-abb8-094645827006 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:31.210781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd72832-d653-467b-abb8-094645827006 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:31.212136 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd72832-d653-467b-abb8-094645827006 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:31.226052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 518/1027] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:31.232573 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec40c6f-88af-4727-aa0f-64fdb2f2f192 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:31.368771 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec40c6f-88af-4727-aa0f-64fdb2f2f192 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:31.379527 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c7ea5-fa3a-418b-9076-e5b770b1c146 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:31.379859 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c7ea5-fa3a-418b-9076-e5b770b1c146 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:31.380105 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c7ea5-fa3a-418b-9076-e5b770b1c146 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:31.382409 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 519/1028] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:16:31.387449 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1da59-e87d-47c8-9a37-08c4c13ae563 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:31.387747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1da59-e87d-47c8-9a37-08c4c13ae563 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:31.388169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1da59-e87d-47c8-9a37-08c4c13ae563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:31.392613 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:31.406782 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3fc1da59-e87d-47c8-9a37-08c4c13ae563 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:31.407999 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3fc1da59-e87d-47c8-9a37-08c4c13ae563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:31.411206 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf42957-ec2e-40e2-96ca-85b6df9a96df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:31.412254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf42957-ec2e-40e2-96ca-85b6df9a96df None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:31.413048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf42957-ec2e-40e2-96ca-85b6df9a96df None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:31.413340 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf42957-ec2e-40e2-96ca-85b6df9a96df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:31.421960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf42957-ec2e-40e2-96ca-85b6df9a96df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:31.423473 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf42957-ec2e-40e2-96ca-85b6df9a96df None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:31.440146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-edf42957-ec2e-40e2-96ca-85b6df9a96df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:16:31.440146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:31.440146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:31.440146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:31.440146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:31.440146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:31.440146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:31.440146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:31.440146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:31.440146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:31.440146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:31.440146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:31.440146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:16:31.440146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:16:31.440146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:16:31.441263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:16:31.441263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:31.441263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:31.441263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:31.441263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:31.441263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:31.441263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:31.441263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:31.441263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:31.441263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:31.441263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:31.441263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:31.441263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:31.441263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:31.441263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:31.441263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:31.441263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:31.442303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:16:31.442303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:16:31.442303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:16:31.442303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:16:31.442303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:16:31.442303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:16:31.442303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:16:31.442303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:31.442303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 510/1029] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:16:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:16:31.450383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf42957-ec2e-40e2-96ca-85b6df9a96df None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:31.499414 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 520/1030] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:16:31.500036 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea711d9b-3a68-4688-b2ac-d9451d979ef0 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:31.500431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea711d9b-3a68-4688-b2ac-d9451d979ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:31.500713 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea711d9b-3a68-4688-b2ac-d9451d979ef0 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:31.502218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 521/1031] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:31.508596 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56543e4-7a01-477d-95b1-4c92602202d0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:31.508923 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56543e4-7a01-477d-95b1-4c92602202d0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:31.509296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56543e4-7a01-477d-95b1-4c92602202d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:31.527251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b56543e4-7a01-477d-95b1-4c92602202d0 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:31.528669 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b56543e4-7a01-477d-95b1-4c92602202d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:31.568317 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf42957-ec2e-40e2-96ca-85b6df9a96df None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:31.601955 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 522/1032] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:16:31.602467 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd13da2-8a9c-49f9-bcaf-a8ff9fcf758a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:31.602754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd13da2-8a9c-49f9-bcaf-a8ff9fcf758a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:31.602867 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56543e4-7a01-477d-95b1-4c92602202d0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:31.602959 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd13da2-8a9c-49f9-bcaf-a8ff9fcf758a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:31.603187 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd13da2-8a9c-49f9-bcaf-a8ff9fcf758a None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:31.603977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd13da2-8a9c-49f9-bcaf-a8ff9fcf758a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:31.604290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd13da2-8a9c-49f9-bcaf-a8ff9fcf758a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:31.605772 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd13da2-8a9c-49f9-bcaf-a8ff9fcf758a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:31.607955 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:31.624387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 511/1033] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:16:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:31.625796 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd13da2-8a9c-49f9-bcaf-a8ff9fcf758a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:31.668855 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:31.718597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56543e4-7a01-477d-95b1-4c92602202d0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:31.728821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd13da2-8a9c-49f9-bcaf-a8ff9fcf758a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:31.763133 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e21ec-fe0d-4223-8810-9c63055046be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:31.764089 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e21ec-fe0d-4223-8810-9c63055046be None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:31.764377 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e21ec-fe0d-4223-8810-9c63055046be None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:31.764635 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e048bf-ce75-466b-ae2d-f79a8bb89f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:31.764747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e21ec-fe0d-4223-8810-9c63055046be None admin] PATH_INFO: `/v3/services/image` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:31.765299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e048bf-ce75-466b-ae2d-f79a8bb89f0a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:31.765416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e21ec-fe0d-4223-8810-9c63055046be None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:31.765595 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e048bf-ce75-466b-ae2d-f79a8bb89f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:31.765884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e048bf-ce75-466b-ae2d-f79a8bb89f0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:31.766717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e048bf-ce75-466b-ae2d-f79a8bb89f0a None admin] RBAC: Adding query filter params (role.id=fca7293e63b14558bb206aaac4e5d1ca, scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b, user.id=b8751a4e92174d18b7b138392d2579a1) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:31.767028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e048bf-ce75-466b-ae2d-f79a8bb89f0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:31.767287 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e21ec-fe0d-4223-8810-9c63055046be None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:31.768458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e048bf-ce75-466b-ae2d-f79a8bb89f0a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:31.775909 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 512/1034] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:16:31 2022] GET /identity/v3/role_assignments?user.id=b8751a4e92174d18b7b138392d2579a1&scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b&role.id=fca7293e63b14558bb206aaac4e5d1ca => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:31.778461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e048bf-ce75-466b-ae2d-f79a8bb89f0a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:31.782206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-818e21ec-fe0d-4223-8810-9c63055046be None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 18:16:31.782206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:31.782206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:16:31.782206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:16:31.782206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:16:31.782206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:16:31.782206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:16:31.782206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:31.782206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 18:16:31.782206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:31.782206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:16:31.782206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:31.782206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:31.782206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:31.782206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:31.782206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:31.782206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:31.783703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:31.783703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:31.783703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:31.783703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:31.783703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:31.783703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:31.783703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:16:31.783703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:16:31.783703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:16:31.783703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:16:31.783703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:31.783703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:31.783703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:31.783703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:31.783703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:31.783703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:31.785277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:31.785277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:31.785277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:31.785277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:31.785277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:31.785277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:31.785277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:31.785277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:31.785277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:31.785277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:31.785277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:16:31.785277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:31.785277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 18:16:31.785277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:31.785277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 523/1035] 10.222.0.33 () {56 vars in 1105 bytes} [Thu Jun 30 18:16:31 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:31.793317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e21ec-fe0d-4223-8810-9c63055046be None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:31.885946 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:31.907777 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e048bf-ce75-466b-ae2d-f79a8bb89f0a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:31.911238 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:31.933940 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e21ec-fe0d-4223-8810-9c63055046be None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:31.952525 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319db0cb-9358-44d0-aae6-cf717048933b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:31.953324 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319db0cb-9358-44d0-aae6-cf717048933b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:31.953642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319db0cb-9358-44d0-aae6-cf717048933b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:31.954164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319db0cb-9358-44d0-aae6-cf717048933b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:31.955398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319db0cb-9358-44d0-aae6-cf717048933b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:31.956858 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319db0cb-9358-44d0-aae6-cf717048933b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:31.958757 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 513/1036] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:31.965615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319db0cb-9358-44d0-aae6-cf717048933b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:31.977446 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8f6b6a-89ef-49c8-b42b-6331485290eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:31.978169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f6b6a-89ef-49c8-b42b-6331485290eb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:31.978444 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f6b6a-89ef-49c8-b42b-6331485290eb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:31.979022 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f6b6a-89ef-49c8-b42b-6331485290eb None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:31.979427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8f6b6a-89ef-49c8-b42b-6331485290eb None admin] RBAC: Adding query filter params (name=image) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:31.979734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8f6b6a-89ef-49c8-b42b-6331485290eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:31.981102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8f6b6a-89ef-49c8-b42b-6331485290eb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:31.991183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 524/1037] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:31.992370 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8f6b6a-89ef-49c8-b42b-6331485290eb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:32.097274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319db0cb-9358-44d0-aae6-cf717048933b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:32.129166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8f6b6a-89ef-49c8-b42b-6331485290eb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:32.150325 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90aa95d-4a0d-4445-a6e6-ab3303bbc528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:32.151284 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90aa95d-4a0d-4445-a6e6-ab3303bbc528 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:32.151569 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90aa95d-4a0d-4445-a6e6-ab3303bbc528 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:32.151845 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90aa95d-4a0d-4445-a6e6-ab3303bbc528 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:32.152559 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90aa95d-4a0d-4445-a6e6-ab3303bbc528 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:32.154035 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90aa95d-4a0d-4445-a6e6-ab3303bbc528 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:32.163257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:32.170822 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cff07-2c26-40e2-8cfe-db074523fdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:32.170822 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cff07-2c26-40e2-8cfe-db074523fdc7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:32.171120 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cff07-2c26-40e2-8cfe-db074523fdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:32.171362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cff07-2c26-40e2-8cfe-db074523fdc7 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:32.172319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619cff07-2c26-40e2-8cfe-db074523fdc7 None admin] RBAC: Adding query filter params (type=image) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:32.172989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619cff07-2c26-40e2-8cfe-db074523fdc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:32.174172 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619cff07-2c26-40e2-8cfe-db074523fdc7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:32.179895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 525/1038] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:32.193724 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 514/1039] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 30 18:16:31 2022] POST /identity/v3/projects => generated 311 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:16:32.194055 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cff07-2c26-40e2-8cfe-db074523fdc7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:32.320516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cff07-2c26-40e2-8cfe-db074523fdc7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:32.375027 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9a4de-366f-4885-a139-14a666cb09a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:32.375719 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9a4de-366f-4885-a139-14a666cb09a3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:32.376277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9a4de-366f-4885-a139-14a666cb09a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:32.376719 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9a4de-366f-4885-a139-14a666cb09a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:32.377923 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9a4de-366f-4885-a139-14a666cb09a3 None admin] RBAC: Adding query filter params (interface=public, service_id=edf77f2682ce40059a254aae64f21d86, region_id=RegionOne) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:32.378332 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9a4de-366f-4885-a139-14a666cb09a3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:32.379839 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9a4de-366f-4885-a139-14a666cb09a3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:32.386498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 526/1040] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 30 18:16:32 2022] GET /identity/v3/endpoints?service_id=edf77f2682ce40059a254aae64f21d86&interface=public®ion_id=RegionOne => generated 194 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:32.402393 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90aa95d-4a0d-4445-a6e6-ab3303bbc528 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:32.434887 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:32.525435 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90aa95d-4a0d-4445-a6e6-ab3303bbc528 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:32.562412 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f09a9-5ebf-4b51-a7f2-9291e70cf0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:32.563080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f09a9-5ebf-4b51-a7f2-9291e70cf0ce None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:32.563366 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f09a9-5ebf-4b51-a7f2-9291e70cf0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:32.563647 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f09a9-5ebf-4b51-a7f2-9291e70cf0ce None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:32.564342 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f09a9-5ebf-4b51-a7f2-9291e70cf0ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:32.565756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f09a9-5ebf-4b51-a7f2-9291e70cf0ce None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:32.574132 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 515/1041] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 30 18:16:32 2022] GET /identity/v3/services => generated 1957 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:16:33.846029 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcd871-99bc-4ddb-8554-00f1aaad4165 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:33.846598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcd871-99bc-4ddb-8554-00f1aaad4165 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:33.846598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcd871-99bc-4ddb-8554-00f1aaad4165 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:33.848189 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 527/1042] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:33.854055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918438de-b4b6-4efe-8c4d-d808df5e41db None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:33.854561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918438de-b4b6-4efe-8c4d-d808df5e41db None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:33.854718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918438de-b4b6-4efe-8c4d-d808df5e41db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:33.872340 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-918438de-b4b6-4efe-8c4d-d808df5e41db None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:33.873540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-918438de-b4b6-4efe-8c4d-d808df5e41db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:33.961627 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 516/1043] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:33.984001 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcd871-99bc-4ddb-8554-00f1aaad4165 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:34.052149 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:34.108666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcd871-99bc-4ddb-8554-00f1aaad4165 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:34.183660 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c53be-a97d-4e6d-ad5f-fc2e050c4942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:34.184927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c53be-a97d-4e6d-ad5f-fc2e050c4942 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:34.185450 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c53be-a97d-4e6d-ad5f-fc2e050c4942 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:34.185996 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c53be-a97d-4e6d-ad5f-fc2e050c4942 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:34.204848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444c53be-a97d-4e6d-ad5f-fc2e050c4942 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:16:34.217116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-444c53be-a97d-4e6d-ad5f-fc2e050c4942 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 18:16:34.217116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:34.217116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:34.217116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:34.217116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:34.217116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:34.217116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:34.217116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:34.217116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:34.217116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:34.217116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:34.217116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:34.217116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:16:34.217116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:16:34.217116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:16:34.217116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:16:34.218199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:16:34.218199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:16:34.218199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:16:34.218199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:16:34.218199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:34.218199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:34.218199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:34.218199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:34.218199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:34.218199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:34.218199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:34.218199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:34.218199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:34.218199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:34.218199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:34.218199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:34.219103 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:34.219103 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:34.219103 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:34.219103 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:34.219103 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:16:34.219103 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:16:34.219103 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:16:34.219103 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:16:34.219103 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:16:34.219103 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:16:34.219103 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 18:16:34.219103 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:34.219741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 528/1044] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Jun 30 18:16:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:34.227085 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918438de-b4b6-4efe-8c4d-d808df5e41db None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:34.266019 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:34.387286 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918438de-b4b6-4efe-8c4d-d808df5e41db None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:34.434176 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ac1f5-72df-4b7b-9182-91f163c98633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:34.435522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ac1f5-72df-4b7b-9182-91f163c98633 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:34.435522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ac1f5-72df-4b7b-9182-91f163c98633 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:34.435522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ac1f5-72df-4b7b-9182-91f163c98633 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:34.436291 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ac1f5-72df-4b7b-9182-91f163c98633 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:34.436487 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ac1f5-72df-4b7b-9182-91f163c98633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:34.437911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ac1f5-72df-4b7b-9182-91f163c98633 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:34.439695 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da3448-2bee-461a-b5e7-04893fc3fab6 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:34.439994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da3448-2bee-461a-b5e7-04893fc3fab6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:34.440283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da3448-2bee-461a-b5e7-04893fc3fab6 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:34.441932 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 529/1045] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:34.448684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369dda1-61f8-42f4-96f1-4aca28c8491e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:34.449027 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369dda1-61f8-42f4-96f1-4aca28c8491e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:34.449501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369dda1-61f8-42f4-96f1-4aca28c8491e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:34.455331 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 517/1046] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 30 18:16:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:34.456160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ac1f5-72df-4b7b-9182-91f163c98633 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:34.470610 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1369dda1-61f8-42f4-96f1-4aca28c8491e None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:34.472418 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1369dda1-61f8-42f4-96f1-4aca28c8491e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:34.587864 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 530/1047] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:34.594166 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:34.613427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369dda1-61f8-42f4-96f1-4aca28c8491e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:34.623005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ac1f5-72df-4b7b-9182-91f163c98633 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:34.670473 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:34.675041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3774d899-0609-4f83-8e8b-bd533c565cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:34.675852 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774d899-0609-4f83-8e8b-bd533c565cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:34.676204 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774d899-0609-4f83-8e8b-bd533c565cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:34.676878 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774d899-0609-4f83-8e8b-bd533c565cc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:34.688913 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3774d899-0609-4f83-8e8b-bd533c565cc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:34.690416 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3774d899-0609-4f83-8e8b-bd533c565cc3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:34.711383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-3774d899-0609-4f83-8e8b-bd533c565cc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:34.711383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:34.711383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:34.711383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:34.711383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:34.711383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:34.711383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:34.711383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:34.711383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:34.711383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:34.711383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:34.711383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:34.711383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:16:34.711383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:16:34.711383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:16:34.712819 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:16:34.712819 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:34.712819 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:34.712819 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:34.712819 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:34.712819 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:34.712819 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:34.712819 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:34.712819 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:34.712819 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:34.712819 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:34.712819 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:34.712819 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:34.712819 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:34.712819 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:34.712819 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:34.712819 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:34.713989 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:16:34.713989 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:16:34.713989 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:34.713989 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:34.717022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 518/1048] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:16:34.718679 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58df382-28b2-4573-a1d7-2db886895437 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:34.718976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58df382-28b2-4573-a1d7-2db886895437 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:34.719261 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58df382-28b2-4573-a1d7-2db886895437 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:34.720621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 519/1049] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:34.721382 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58df382-28b2-4573-a1d7-2db886895437 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:34.736400 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369dda1-61f8-42f4-96f1-4aca28c8491e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:34.772165 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d396d7-85fb-4b65-9f50-5b2a769e5209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:34.772923 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d396d7-85fb-4b65-9f50-5b2a769e5209 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:34.773210 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d396d7-85fb-4b65-9f50-5b2a769e5209 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:34.773494 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d396d7-85fb-4b65-9f50-5b2a769e5209 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:34.775085 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d396d7-85fb-4b65-9f50-5b2a769e5209 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:34.776222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d396d7-85fb-4b65-9f50-5b2a769e5209 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:34.777892 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 531/1050] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:34.779062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb109a-3b3e-401f-998d-5a085164a2b8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:34.779438 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb109a-3b3e-401f-998d-5a085164a2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:34.779634 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb109a-3b3e-401f-998d-5a085164a2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:34.798687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c0eb109a-3b3e-401f-998d-5a085164a2b8 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:34.799895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c0eb109a-3b3e-401f-998d-5a085164a2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:34.828822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58df382-28b2-4573-a1d7-2db886895437 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:34.859824 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:34.866912 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0128fe-7c78-474a-9438-34d4cc84d52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:34.867633 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0128fe-7c78-474a-9438-34d4cc84d52a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:34.868012 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0128fe-7c78-474a-9438-34d4cc84d52a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:34.868217 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0128fe-7c78-474a-9438-34d4cc84d52a None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:34.868957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0128fe-7c78-474a-9438-34d4cc84d52a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:34.869273 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0128fe-7c78-474a-9438-34d4cc84d52a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:34.869783 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 532/1051] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:34.870379 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb109a-3b3e-401f-998d-5a085164a2b8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:34.870720 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0128fe-7c78-474a-9438-34d4cc84d52a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:34.884844 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 520/1052] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:34.890663 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0128fe-7c78-474a-9438-34d4cc84d52a None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:34.928212 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:34.983416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb109a-3b3e-401f-998d-5a085164a2b8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:34.995188 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0128fe-7c78-474a-9438-34d4cc84d52a None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:35.017115 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8b4ac-1843-454f-a420-df286e4d6a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:35.017807 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8b4ac-1843-454f-a420-df286e4d6a66 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:35.018095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8b4ac-1843-454f-a420-df286e4d6a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:35.018376 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8b4ac-1843-454f-a420-df286e4d6a66 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:35.019076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8b4ac-1843-454f-a420-df286e4d6a66 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:35.020520 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8b4ac-1843-454f-a420-df286e4d6a66 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:35.030804 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce82d5-8c32-4fce-867b-d27377b7b24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:35.031692 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce82d5-8c32-4fce-867b-d27377b7b24e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:35.032036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce82d5-8c32-4fce-867b-d27377b7b24e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:35.032352 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce82d5-8c32-4fce-867b-d27377b7b24e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:35.041426 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce82d5-8c32-4fce-867b-d27377b7b24e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:35.042923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce82d5-8c32-4fce-867b-d27377b7b24e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:35.043816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 533/1053] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 30 18:16:34 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:16:35.044476 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8b4ac-1843-454f-a420-df286e4d6a66 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:35.068337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-1dce82d5-8c32-4fce-867b-d27377b7b24e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 18:16:35.068337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:35.068337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:35.068337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:35.068337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:35.068337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:35.068337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:35.068337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:35.068337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:35.068337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:35.068337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:35.068337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:35.068337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:16:35.068337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:16:35.068337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:16:35.068337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:16:35.069375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:35.069375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:35.069375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:16:35.069375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:16:35.069375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:16:35.069375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:16:35.069375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:35.069375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:35.069375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:35.069375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:35.069375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:35.069375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:35.069375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:35.069375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:35.069375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:35.069375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:35.069375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:35.069375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:35.071341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:35.071341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:35.071341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:16:35.071341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:16:35.071341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:16:35.071341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:16:35.071341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:16:35.071341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:16:35.071341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 18:16:35.071341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:35.071341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 521/1054] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 30 18:16:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:16:35.076012 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce82d5-8c32-4fce-867b-d27377b7b24e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:35.137828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:35.166026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8b4ac-1843-454f-a420-df286e4d6a66 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:35.167868 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:35.197513 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce82d5-8c32-4fce-867b-d27377b7b24e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:35.206405 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637fa6fd-5b09-4a71-94cf-6d1abb1b7b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:35.207205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fa6fd-5b09-4a71-94cf-6d1abb1b7b67 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:35.207489 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fa6fd-5b09-4a71-94cf-6d1abb1b7b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:35.207763 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fa6fd-5b09-4a71-94cf-6d1abb1b7b67 None admin] PATH_INFO: `/v3/services/image` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:35.208410 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637fa6fd-5b09-4a71-94cf-6d1abb1b7b67 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:35.209932 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637fa6fd-5b09-4a71-94cf-6d1abb1b7b67 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:35.221706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-637fa6fd-5b09-4a71-94cf-6d1abb1b7b67 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 18:16:35.221706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:35.221706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:16:35.221706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:16:35.221706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:16:35.221706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:16:35.221706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:16:35.221706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:35.221706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 18:16:35.221706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:35.221706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:16:35.221706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:35.221706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:35.221706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:35.221706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:35.221706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:35.221706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:35.223628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:35.223628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:35.223628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:35.223628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:35.223628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:35.223628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:35.223628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:16:35.223628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:16:35.223628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:16:35.223628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:16:35.223628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:35.223628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:35.223628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:35.223628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:35.223628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:35.223628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:35.225257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:35.225257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:35.225257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:35.225257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:35.225257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:35.225257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:35.225257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:35.225257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:35.225257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:35.225257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:35.225257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:16:35.225257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:16:35.225257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 18:16:35.225257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:35.225257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 534/1055] 10.222.0.33 () {56 vars in 1105 bytes} [Thu Jun 30 18:16:35 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:35.231961 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637fa6fd-5b09-4a71-94cf-6d1abb1b7b67 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:35.242209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47024e99-4ffb-4625-b401-b78bc496ead1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:35.242893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47024e99-4ffb-4625-b401-b78bc496ead1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:35.243178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47024e99-4ffb-4625-b401-b78bc496ead1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:35.243458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47024e99-4ffb-4625-b401-b78bc496ead1 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:35.244307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47024e99-4ffb-4625-b401-b78bc496ead1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:35.244621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47024e99-4ffb-4625-b401-b78bc496ead1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:35.246125 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47024e99-4ffb-4625-b401-b78bc496ead1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:35.273713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 522/1056] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 30 18:16:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:35.275869 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47024e99-4ffb-4625-b401-b78bc496ead1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:35.350007 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637fa6fd-5b09-4a71-94cf-6d1abb1b7b67 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:35.388271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb88912-db6d-4193-a4cd-fc94baa65785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:35.388944 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb88912-db6d-4193-a4cd-fc94baa65785 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:35.389234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb88912-db6d-4193-a4cd-fc94baa65785 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:35.389516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb88912-db6d-4193-a4cd-fc94baa65785 None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:35.390261 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb88912-db6d-4193-a4cd-fc94baa65785 None admin] RBAC: Adding query filter params (name=image) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:35.390569 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb88912-db6d-4193-a4cd-fc94baa65785 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:35.391946 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb88912-db6d-4193-a4cd-fc94baa65785 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:35.396664 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47024e99-4ffb-4625-b401-b78bc496ead1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:35.397569 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 535/1057] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:35.402933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb88912-db6d-4193-a4cd-fc94baa65785 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:35.409119 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:35.436119 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:35.437778 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb4c5d-5aac-48b9-ad0f-28a9e46b8686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:35.438571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb4c5d-5aac-48b9-ad0f-28a9e46b8686 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:35.438903 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb4c5d-5aac-48b9-ad0f-28a9e46b8686 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:35.439175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb4c5d-5aac-48b9-ad0f-28a9e46b8686 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:35.451401 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb4c5d-5aac-48b9-ad0f-28a9e46b8686 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:35.452869 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb4c5d-5aac-48b9-ad0f-28a9e46b8686 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:35.475775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-1dfb4c5d-5aac-48b9-ad0f-28a9e46b8686 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:35.475775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:35.475775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:35.475775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:35.475775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:35.475775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:35.475775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:35.475775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:35.475775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:35.475775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:35.475775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:35.475775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:35.475775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:16:35.475775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:16:35.475775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:16:35.477117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:16:35.477117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:35.477117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:35.477117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:35.477117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:35.477117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:35.477117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:35.477117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:35.477117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:35.477117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:35.477117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:35.477117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:35.477117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:35.477117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:35.477117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:35.477117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:35.477117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:35.478267 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:16:35.478267 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:16:35.478267 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:35.478267 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:35.478267 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 523/1058] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:16:35.482973 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb4c5d-5aac-48b9-ad0f-28a9e46b8686 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:35.521149 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb88912-db6d-4193-a4cd-fc94baa65785 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:35.569702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036d88a-ab12-41b3-8834-bce02380b9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:35.570557 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036d88a-ab12-41b3-8834-bce02380b9cc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:35.571036 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036d88a-ab12-41b3-8834-bce02380b9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:35.571484 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036d88a-ab12-41b3-8834-bce02380b9cc None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:35.572634 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036d88a-ab12-41b3-8834-bce02380b9cc None admin] RBAC: Adding query filter params (type=image) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:35.573127 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036d88a-ab12-41b3-8834-bce02380b9cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:35.575214 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036d88a-ab12-41b3-8834-bce02380b9cc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:35.581361 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 536/1059] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:35.587906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036d88a-ab12-41b3-8834-bce02380b9cc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:35.601838 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb4c5d-5aac-48b9-ad0f-28a9e46b8686 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:35.642778 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef31d8-f49c-4f05-8a17-df2be129e6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:35.643451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef31d8-f49c-4f05-8a17-df2be129e6f5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:35.643742 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef31d8-f49c-4f05-8a17-df2be129e6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:35.644020 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef31d8-f49c-4f05-8a17-df2be129e6f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:35.644747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef31d8-f49c-4f05-8a17-df2be129e6f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:35.645057 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef31d8-f49c-4f05-8a17-df2be129e6f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:35.647142 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef31d8-f49c-4f05-8a17-df2be129e6f5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:35.665525 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 524/1060] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:16:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:35.666726 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef31d8-f49c-4f05-8a17-df2be129e6f5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:35.688202 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:35.693126 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:35.697779 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036d88a-ab12-41b3-8834-bce02380b9cc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:35.732635 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c95b9-dd3c-44f8-876b-574a95583f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:35.733216 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c95b9-dd3c-44f8-876b-574a95583f5e None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:35.733508 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c95b9-dd3c-44f8-876b-574a95583f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:35.733826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c95b9-dd3c-44f8-876b-574a95583f5e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:35.734570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c95b9-dd3c-44f8-876b-574a95583f5e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:35.735959 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c95b9-dd3c-44f8-876b-574a95583f5e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:35.769619 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 537/1061] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 30 18:16:35 2022] POST /identity/v3/endpoints => generated 327 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:16:35.799762 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef31d8-f49c-4f05-8a17-df2be129e6f5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:35.859119 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e8bcf8-8a69-477f-92a2-438bea87b394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:35.859952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8bcf8-8a69-477f-92a2-438bea87b394 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:35.860767 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8bcf8-8a69-477f-92a2-438bea87b394 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:35.860767 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8bcf8-8a69-477f-92a2-438bea87b394 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:35.873703 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e8bcf8-8a69-477f-92a2-438bea87b394 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:35.875837 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e8bcf8-8a69-477f-92a2-438bea87b394 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:35.901727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-50e8bcf8-8a69-477f-92a2-438bea87b394 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:16:35.901727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:35.901727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:35.901727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:35.901727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:35.901727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:35.901727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:35.901727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:35.901727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:35.901727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:35.901727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:35.901727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:35.901727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:16:35.901727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:16:35.901727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:16:35.902747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:16:35.902747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:35.902747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:35.902747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:35.902747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:35.902747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:35.902747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:35.902747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:35.902747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:35.902747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:35.902747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:35.902747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:35.902747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:35.902747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:35.902747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:35.902747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:35.902747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:35.903910 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:16:35.903910 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:16:35.903910 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:16:35.903910 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:16:35.903910 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:16:35.903910 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:16:35.903910 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:16:35.903910 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:35.905947 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 525/1062] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:16:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:35.908918 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c95b9-dd3c-44f8-876b-574a95583f5e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:36.030190 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c95b9-dd3c-44f8-876b-574a95583f5e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:36.078839 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037edf8-7ae7-4294-9d71-fbd0e1d8ab82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:36.079501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037edf8-7ae7-4294-9d71-fbd0e1d8ab82 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:36.079784 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037edf8-7ae7-4294-9d71-fbd0e1d8ab82 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:36.080058 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037edf8-7ae7-4294-9d71-fbd0e1d8ab82 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:36.080839 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a037edf8-7ae7-4294-9d71-fbd0e1d8ab82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:36.081149 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a037edf8-7ae7-4294-9d71-fbd0e1d8ab82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:36.082608 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a037edf8-7ae7-4294-9d71-fbd0e1d8ab82 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:36.096433 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 538/1063] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:16:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:36.102224 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e8bcf8-8a69-477f-92a2-438bea87b394 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:36.184069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:36.209639 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e8bcf8-8a69-477f-92a2-438bea87b394 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:36.248549 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3c69a-8bf6-49c9-9c48-1af45d785c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:36.249443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3c69a-8bf6-49c9-9c48-1af45d785c1a None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:36.249765 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3c69a-8bf6-49c9-9c48-1af45d785c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:36.250015 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3c69a-8bf6-49c9-9c48-1af45d785c1a None admin] PATH_INFO: `/v3/projects/8c7e1ee12aa14b7ab13e07822826b25b/users/b8751a4e92174d18b7b138392d2579a1/roles/fca7293e63b14558bb206aaac4e5d1ca` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:36.252375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3c69a-8bf6-49c9-9c48-1af45d785c1a None admin] RBAC: Authorizing `identity:create_grant(project_id=8c7e1ee12aa14b7ab13e07822826b25b, user_id=b8751a4e92174d18b7b138392d2579a1, role_id=fca7293e63b14558bb206aaac4e5d1ca)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:36.253892 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3c69a-8bf6-49c9-9c48-1af45d785c1a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:36.272622 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 526/1064] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:16:36 2022] PUT /identity/v3/projects/8c7e1ee12aa14b7ab13e07822826b25b/users/b8751a4e92174d18b7b138392d2579a1/roles/fca7293e63b14558bb206aaac4e5d1ca => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:16:37.253824 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ee7b7-048e-4f54-891e-aaff8773d8fb None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:37.254431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ee7b7-048e-4f54-891e-aaff8773d8fb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:37.254431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ee7b7-048e-4f54-891e-aaff8773d8fb None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:37.256169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 539/1065] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:37.262508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa65838-394b-4dbb-9ab9-81a0db520dad None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:37.263222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa65838-394b-4dbb-9ab9-81a0db520dad None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:37.263222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa65838-394b-4dbb-9ab9-81a0db520dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:37.282169 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9aa65838-394b-4dbb-9ab9-81a0db520dad None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:37.283366 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9aa65838-394b-4dbb-9ab9-81a0db520dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:37.360528 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 527/1066] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:37.383765 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ee7b7-048e-4f54-891e-aaff8773d8fb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:37.399448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:37.500880 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ee7b7-048e-4f54-891e-aaff8773d8fb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:37.537811 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf5211-c6a1-468a-8323-462999e28ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:37.538851 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf5211-c6a1-468a-8323-462999e28ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:37.539291 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf5211-c6a1-468a-8323-462999e28ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:37.539743 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf5211-c6a1-468a-8323-462999e28ba0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:37.552681 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf5211-c6a1-468a-8323-462999e28ba0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:16:37.561301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-0fdf5211-c6a1-468a-8323-462999e28ba0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:16:37.561301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:37.561301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:37.561301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:37.561301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:37.561301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:37.561301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:37.561301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:37.561301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:37.561301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:37.561301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:37.561301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:37.561301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:16:37.561301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:16:37.561301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:16:37.561301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:16:37.562385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:16:37.562385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:16:37.562385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:16:37.562385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:16:37.562385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:37.562385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:37.562385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:37.562385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:37.562385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:37.562385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:37.562385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:37.562385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:37.562385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:37.562385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:37.562385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:37.562385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:37.563252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:37.563252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:37.563252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:37.563252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:37.563252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:16:37.563252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:16:37.563252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:16:37.563252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:16:37.563252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:16:37.563252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:16:37.563252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:16:37.563252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:37.565733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 540/1067] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:16:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:37.569791 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa65838-394b-4dbb-9ab9-81a0db520dad None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:37.641136 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:37.680140 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa65838-394b-4dbb-9ab9-81a0db520dad None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:37.715842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208da64-5204-4382-ac84-f055f6ec204d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:37.716510 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208da64-5204-4382-ac84-f055f6ec204d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:37.716794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208da64-5204-4382-ac84-f055f6ec204d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:37.717070 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208da64-5204-4382-ac84-f055f6ec204d None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:37.717847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208da64-5204-4382-ac84-f055f6ec204d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:37.718158 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208da64-5204-4382-ac84-f055f6ec204d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:37.719540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208da64-5204-4382-ac84-f055f6ec204d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:37.733396 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 528/1068] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:16:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:37.734734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf5211-c6a1-468a-8323-462999e28ba0 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:37.813711 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:37.847183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf5211-c6a1-468a-8323-462999e28ba0 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:37.883748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b2c35-fd76-4173-b2ae-03bb5e9b55e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:37.884697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b2c35-fd76-4173-b2ae-03bb5e9b55e2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:37.884990 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b2c35-fd76-4173-b2ae-03bb5e9b55e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:37.885272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b2c35-fd76-4173-b2ae-03bb5e9b55e2 None admin] PATH_INFO: `/v3/users/98c171b0c6cb4717af16f8f8fc939a9d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:37.899991 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5b2c35-fd76-4173-b2ae-03bb5e9b55e2 None admin] RBAC: Authorizing `identity:get_user(user_id=98c171b0c6cb4717af16f8f8fc939a9d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:37.901447 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5b2c35-fd76-4173-b2ae-03bb5e9b55e2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:37.907952 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 541/1069] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:16:37 2022] GET /identity/v3/users/98c171b0c6cb4717af16f8f8fc939a9d => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:37.909024 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208da64-5204-4382-ac84-f055f6ec204d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:37.951015 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29eb4e-7dae-46bb-a61a-d62b609ecf85 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:37.951339 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29eb4e-7dae-46bb-a61a-d62b609ecf85 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:37.951584 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29eb4e-7dae-46bb-a61a-d62b609ecf85 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:37.952956 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 542/1070] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:16:37.957978 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea2d3d-030a-4ac1-ab7e-64d16d0443e4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:37.958261 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea2d3d-030a-4ac1-ab7e-64d16d0443e4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:37.958539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea2d3d-030a-4ac1-ab7e-64d16d0443e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:37.977557 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e7ea2d3d-030a-4ac1-ab7e-64d16d0443e4 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:37.978854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e7ea2d3d-030a-4ac1-ab7e-64d16d0443e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:38.026586 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208da64-5204-4382-ac84-f055f6ec204d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:38.057810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:38.061633 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 543/1071] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:16:38.066313 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612036f-e78b-44e5-8d82-1a05c128e95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:38.066830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612036f-e78b-44e5-8d82-1a05c128e95c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:38.067173 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612036f-e78b-44e5-8d82-1a05c128e95c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:38.067789 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612036f-e78b-44e5-8d82-1a05c128e95c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:38.081429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea2d3d-030a-4ac1-ab7e-64d16d0443e4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:38.143112 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a612036f-e78b-44e5-8d82-1a05c128e95c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:38.144792 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:38.145055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a612036f-e78b-44e5-8d82-1a05c128e95c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:38.191565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea2d3d-030a-4ac1-ab7e-64d16d0443e4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:38.238957 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747c7e1-8edd-4a6a-a629-2a7f960ced89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:38.240757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747c7e1-8edd-4a6a-a629-2a7f960ced89 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:38.241030 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747c7e1-8edd-4a6a-a629-2a7f960ced89 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:38.241317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747c7e1-8edd-4a6a-a629-2a7f960ced89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:38.277614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 529/1072] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:16:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 18:16:38.279825 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612036f-e78b-44e5-8d82-1a05c128e95c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:38.326275 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:38.346159 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d747c7e1-8edd-4a6a-a629-2a7f960ced89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:38.347629 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d747c7e1-8edd-4a6a-a629-2a7f960ced89 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:38.404498 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612036f-e78b-44e5-8d82-1a05c128e95c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:38.409617 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:38.445027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c82e8-5923-4177-8406-6209d3c2c910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:38.445027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c82e8-5923-4177-8406-6209d3c2c910 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:38.445690 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c82e8-5923-4177-8406-6209d3c2c910 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:38.445690 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c82e8-5923-4177-8406-6209d3c2c910 None admin] PATH_INFO: `/v3/projects/f322a75bee2e40dfb392011793ec5104` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:38.446702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c82e8-5923-4177-8406-6209d3c2c910 None admin] RBAC: Authorizing `identity:get_project(project_id=f322a75bee2e40dfb392011793ec5104)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:38.448267 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c82e8-5923-4177-8406-6209d3c2c910 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:38.450052 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 530/1073] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:16:38 2022] GET /identity/v3/projects/f322a75bee2e40dfb392011793ec5104 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:38.451295 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41deeb17-83b2-4aed-95b5-eec6f26df05c None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:38.451601 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41deeb17-83b2-4aed-95b5-eec6f26df05c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:38.451884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41deeb17-83b2-4aed-95b5-eec6f26df05c None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:38.453128 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 531/1074] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:38.456402 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41deeb17-83b2-4aed-95b5-eec6f26df05c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:38.471757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 544/1075] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:16:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 18:16:38.471757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a85e8e-e898-4b03-bcb4-3978595e96df None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:38.471757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a85e8e-e898-4b03-bcb4-3978595e96df None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:38.471757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a85e8e-e898-4b03-bcb4-3978595e96df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:38.484986 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-35a85e8e-e898-4b03-bcb4-3978595e96df None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:38.486244 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-35a85e8e-e898-4b03-bcb4-3978595e96df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:38.553571 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 545/1076] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:38.554460 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a85e8e-e898-4b03-bcb4-3978595e96df None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:38.556573 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:38.565522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41deeb17-83b2-4aed-95b5-eec6f26df05c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:38.604018 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5bacc-01f4-45c3-b045-eae3f3bf87f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:38.604579 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5bacc-01f4-45c3-b045-eae3f3bf87f8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:38.604869 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5bacc-01f4-45c3-b045-eae3f3bf87f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:38.605158 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5bacc-01f4-45c3-b045-eae3f3bf87f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:38.606034 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5bacc-01f4-45c3-b045-eae3f3bf87f8 None admin] RBAC: Adding query filter params (role.id=0ef3687af20a443281e3cfdceca3597d, scope.project.id=f322a75bee2e40dfb392011793ec5104, user.id=98c171b0c6cb4717af16f8f8fc939a9d) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:38.606352 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5bacc-01f4-45c3-b045-eae3f3bf87f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:38.607804 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5bacc-01f4-45c3-b045-eae3f3bf87f8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:38.614166 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 532/1077] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:16:38 2022] GET /identity/v3/role_assignments?user.id=98c171b0c6cb4717af16f8f8fc939a9d&scope.project.id=f322a75bee2e40dfb392011793ec5104&role.id=0ef3687af20a443281e3cfdceca3597d => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:38.614849 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5bacc-01f4-45c3-b045-eae3f3bf87f8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:38.656865 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:38.674041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a85e8e-e898-4b03-bcb4-3978595e96df None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:38.718409 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f5c118-8be8-4a84-bd72-85b0764a97e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:38.719199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5c118-8be8-4a84-bd72-85b0764a97e3 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:38.719486 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5c118-8be8-4a84-bd72-85b0764a97e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:38.719772 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f5c118-8be8-4a84-bd72-85b0764a97e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:38.731771 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f5c118-8be8-4a84-bd72-85b0764a97e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:38.732109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5bacc-01f4-45c3-b045-eae3f3bf87f8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:38.733255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f5c118-8be8-4a84-bd72-85b0764a97e3 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:38.753984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-09f5c118-8be8-4a84-bd72-85b0764a97e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:38.753984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:38.753984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:38.753984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:38.753984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:38.753984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:38.753984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:38.753984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:38.753984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:38.753984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:38.753984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:38.753984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:38.753984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:16:38.753984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:16:38.753984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:16:38.755050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:16:38.755050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:38.755050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:38.755050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:38.755050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:38.755050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:38.755050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:38.755050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:38.755050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:38.755050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:38.755050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:38.755050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:38.755050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:38.755050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:38.755050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:38.755050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:38.755050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:38.756522 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:16:38.756522 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:16:38.756522 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:38.756522 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:38.761628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 546/1078] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:38.762366 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f5c118-8be8-4a84-bd72-85b0764a97e3 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:38.775397 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821b77b-0c96-4634-bdaf-20c535b15f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:38.776644 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821b77b-0c96-4634-bdaf-20c535b15f5e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:38.776895 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821b77b-0c96-4634-bdaf-20c535b15f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:38.777079 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821b77b-0c96-4634-bdaf-20c535b15f5e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:38.788307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c821b77b-0c96-4634-bdaf-20c535b15f5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:16:38.796518 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c821b77b-0c96-4634-bdaf-20c535b15f5e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 18:16:38.796518 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:38.796518 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:38.796518 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:38.796518 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:38.796518 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:38.796518 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:38.796518 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:38.796518 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:38.796518 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:38.796518 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:38.796518 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:38.796518 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:16:38.796518 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:16:38.796518 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:16:38.796518 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:16:38.797729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:16:38.797729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:16:38.797729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:16:38.797729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:16:38.797729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:38.797729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:38.797729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:38.797729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:38.797729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:38.797729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:38.797729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:38.797729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:38.797729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:38.797729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:38.797729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:38.797729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:38.798591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:38.798591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:38.798591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:38.798591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:38.798591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:16:38.798591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:16:38.798591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:16:38.798591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:16:38.798591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:16:38.798591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:16:38.798591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 18:16:38.798591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:38.798591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 533/1079] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Jun 30 18:16:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:38.804448 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821b77b-0c96-4634-bdaf-20c535b15f5e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:38.831449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:38.872848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f5c118-8be8-4a84-bd72-85b0764a97e3 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:38.908180 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb64fa-ae23-4d56-acb6-a5c823b2700a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:38.908840 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb64fa-ae23-4d56-acb6-a5c823b2700a None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:38.909126 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb64fa-ae23-4d56-acb6-a5c823b2700a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:38.909404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb64fa-ae23-4d56-acb6-a5c823b2700a None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:38.910157 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb64fa-ae23-4d56-acb6-a5c823b2700a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:38.910464 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb64fa-ae23-4d56-acb6-a5c823b2700a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:38.911866 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb64fa-ae23-4d56-acb6-a5c823b2700a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:38.917488 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821b77b-0c96-4634-bdaf-20c535b15f5e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:38.921175 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:38.926099 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 547/1080] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:38.966185 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37e403-d361-4cc1-b6a8-cab6ba6bd33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:38.966864 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37e403-d361-4cc1-b6a8-cab6ba6bd33b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:38.967145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37e403-d361-4cc1-b6a8-cab6ba6bd33b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:38.967417 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37e403-d361-4cc1-b6a8-cab6ba6bd33b None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:38.968171 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37e403-d361-4cc1-b6a8-cab6ba6bd33b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:38.968478 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37e403-d361-4cc1-b6a8-cab6ba6bd33b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:38.970009 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37e403-d361-4cc1-b6a8-cab6ba6bd33b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:38.984885 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 534/1081] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 30 18:16:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:38.986718 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb64fa-ae23-4d56-acb6-a5c823b2700a None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:39.110306 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb64fa-ae23-4d56-acb6-a5c823b2700a None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:39.150463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364bc968-66d2-4116-aa4f-bf7f4a31261e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:39.151239 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bc968-66d2-4116-aa4f-bf7f4a31261e None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:39.151525 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bc968-66d2-4116-aa4f-bf7f4a31261e None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:39.151800 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bc968-66d2-4116-aa4f-bf7f4a31261e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:39.162389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364bc968-66d2-4116-aa4f-bf7f4a31261e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:39.163736 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364bc968-66d2-4116-aa4f-bf7f4a31261e None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:39.183437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-364bc968-66d2-4116-aa4f-bf7f4a31261e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:39.183437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:39.183437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:39.183437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:39.183437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:39.183437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:39.183437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:39.183437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:39.183437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:39.183437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:39.183437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:39.183437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:39.183437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:16:39.183437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:16:39.183437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:16:39.184393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:16:39.184393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:39.184393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:39.184393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:39.184393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:39.184393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:39.184393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:39.184393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:39.184393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:39.184393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:39.184393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:39.184393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:39.184393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:39.184393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:39.184393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:39.184393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:39.184393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:39.185279 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:16:39.185279 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:16:39.185279 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:39.185279 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:39.185279 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 548/1082] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:16:39.190617 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37e403-d361-4cc1-b6a8-cab6ba6bd33b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:39.281653 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:39.297482 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37e403-d361-4cc1-b6a8-cab6ba6bd33b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:39.331735 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018ee5b-9829-4dbc-ae26-9395182faf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:39.332359 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018ee5b-9829-4dbc-ae26-9395182faf92 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:39.332640 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018ee5b-9829-4dbc-ae26-9395182faf92 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:39.332914 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018ee5b-9829-4dbc-ae26-9395182faf92 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:39.333831 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018ee5b-9829-4dbc-ae26-9395182faf92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:39.334018 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018ee5b-9829-4dbc-ae26-9395182faf92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:39.335493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018ee5b-9829-4dbc-ae26-9395182faf92 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:39.353951 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 535/1083] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:39.354171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364bc968-66d2-4116-aa4f-bf7f4a31261e None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:39.409706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:39.460528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364bc968-66d2-4116-aa4f-bf7f4a31261e None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:39.498358 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904c616-90a5-4fae-a712-0dd23f6049a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:39.499157 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904c616-90a5-4fae-a712-0dd23f6049a9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:39.499443 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904c616-90a5-4fae-a712-0dd23f6049a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:39.499729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904c616-90a5-4fae-a712-0dd23f6049a9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:39.508635 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f904c616-90a5-4fae-a712-0dd23f6049a9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:39.510151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f904c616-90a5-4fae-a712-0dd23f6049a9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:39.525516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-f904c616-90a5-4fae-a712-0dd23f6049a9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 18:16:39.525516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:39.525516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:39.525516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:39.525516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:39.525516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:39.525516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:39.525516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:39.525516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:39.525516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:39.525516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:39.525516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:39.525516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:16:39.525516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:16:39.525516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:16:39.525516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:16:39.526729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:39.526729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:39.526729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:16:39.526729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:16:39.526729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:16:39.526729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:16:39.526729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:39.526729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:39.526729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:39.526729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:39.526729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:39.526729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:39.526729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:39.526729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:39.526729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:39.526729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:39.526729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:39.526729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:39.528197 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:39.528197 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:39.528197 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:16:39.528197 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:16:39.528197 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:16:39.528197 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:16:39.528197 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:16:39.528197 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:16:39.528197 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 18:16:39.528197 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:16:39.528197 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 549/1084] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 30 18:16:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:39.532541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018ee5b-9829-4dbc-ae26-9395182faf92 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:39.660267 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018ee5b-9829-4dbc-ae26-9395182faf92 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:39.700605 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887b304-c95f-484a-807e-33ec790de88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:39.701272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887b304-c95f-484a-807e-33ec790de88f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:39.701561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887b304-c95f-484a-807e-33ec790de88f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:39.701879 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887b304-c95f-484a-807e-33ec790de88f None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:39.702724 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887b304-c95f-484a-807e-33ec790de88f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:39.703043 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887b304-c95f-484a-807e-33ec790de88f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:39.704492 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887b304-c95f-484a-807e-33ec790de88f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:39.723683 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 536/1085] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 30 18:16:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:39.731153 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904c616-90a5-4fae-a712-0dd23f6049a9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:39.838119 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904c616-90a5-4fae-a712-0dd23f6049a9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:39.854738 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:39.877204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d431e-e2eb-49ed-a906-433b98ef8ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:39.877894 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d431e-e2eb-49ed-a906-433b98ef8ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:39.878186 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d431e-e2eb-49ed-a906-433b98ef8ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:39.878474 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d431e-e2eb-49ed-a906-433b98ef8ce9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:39.967751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d431e-e2eb-49ed-a906-433b98ef8ce9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:39.969214 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d431e-e2eb-49ed-a906-433b98ef8ce9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:40.078184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 550/1086] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:16:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 18:16:40.084767 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887b304-c95f-484a-807e-33ec790de88f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:40.088055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:40.191965 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887b304-c95f-484a-807e-33ec790de88f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:40.227257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c58d2-0727-4c54-b1c2-437dee397a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:40.228145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c58d2-0727-4c54-b1c2-437dee397a85 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:40.228426 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c58d2-0727-4c54-b1c2-437dee397a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:40.228709 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c58d2-0727-4c54-b1c2-437dee397a85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:40.240913 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c58d2-0727-4c54-b1c2-437dee397a85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:40.242505 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c58d2-0727-4c54-b1c2-437dee397a85 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:40.263277 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e72c58d2-0727-4c54-b1c2-437dee397a85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:40.263277 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:40.263277 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:40.263277 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:40.263277 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:40.263277 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:40.263277 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:40.263277 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:40.263277 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:40.263277 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:40.263277 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:40.263277 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:40.263277 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:16:40.263277 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:16:40.263277 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:16:40.264501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:16:40.264501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:40.264501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:40.264501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:40.264501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:40.264501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:40.264501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:40.264501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:40.264501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:40.264501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:40.264501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:40.264501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:40.264501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:40.264501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:40.264501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:40.264501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:40.264501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:40.267311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:16:40.267311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:16:40.267311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:16:40.267311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:40.267311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 537/1087] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:40.271282 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d431e-e2eb-49ed-a906-433b98ef8ce9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:40.294203 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:40.391958 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d431e-e2eb-49ed-a906-433b98ef8ce9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:40.430296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc1fcb7-4a06-4ca0-b860-b8879b97bddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:40.431073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1fcb7-4a06-4ca0-b860-b8879b97bddf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:40.431362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1fcb7-4a06-4ca0-b860-b8879b97bddf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:40.431644 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1fcb7-4a06-4ca0-b860-b8879b97bddf None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:40.432374 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc1fcb7-4a06-4ca0-b860-b8879b97bddf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:40.432684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc1fcb7-4a06-4ca0-b860-b8879b97bddf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:40.434332 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc1fcb7-4a06-4ca0-b860-b8879b97bddf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:40.447790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 551/1088] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:40.453832 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c58d2-0727-4c54-b1c2-437dee397a85 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:40.517504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:40.573032 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c58d2-0727-4c54-b1c2-437dee397a85 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:40.611766 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a20a6b-4fd1-4786-aa83-c1003cc99b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:40.612560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a20a6b-4fd1-4786-aa83-c1003cc99b23 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:40.612844 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a20a6b-4fd1-4786-aa83-c1003cc99b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:40.613119 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a20a6b-4fd1-4786-aa83-c1003cc99b23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:40.621483 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a20a6b-4fd1-4786-aa83-c1003cc99b23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:40.623039 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a20a6b-4fd1-4786-aa83-c1003cc99b23 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:40.638235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-57a20a6b-4fd1-4786-aa83-c1003cc99b23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:16:40.638235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:40.638235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:40.638235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:40.638235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:40.638235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:40.638235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:40.638235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:40.638235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:40.638235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:40.638235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:40.638235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:40.638235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:16:40.638235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:16:40.638235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:16:40.639621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:16:40.639621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:40.639621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:40.639621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:40.639621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:40.639621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:40.639621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:40.639621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:40.639621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:40.639621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:40.639621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:40.639621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:40.639621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:40.639621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:40.639621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:40.639621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:40.639621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:40.640738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:16:40.640738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:16:40.640738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:16:40.640738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:16:40.640738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:16:40.640738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:16:40.640738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:16:40.640738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:40.641780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 538/1089] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:16:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:16:40.646627 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc1fcb7-4a06-4ca0-b860-b8879b97bddf None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:40.679044 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbfc7e-940d-47e1-9773-c4643b91671b None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:40.679427 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbfc7e-940d-47e1-9773-c4643b91671b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:40.679590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbfc7e-940d-47e1-9773-c4643b91671b None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:40.681027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 539/1090] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:40.686913 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5478eca-5ff3-4a1c-baa8-89007295f104 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:40.687241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5478eca-5ff3-4a1c-baa8-89007295f104 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:40.687568 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5478eca-5ff3-4a1c-baa8-89007295f104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:40.706145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c5478eca-5ff3-4a1c-baa8-89007295f104 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:40.707430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c5478eca-5ff3-4a1c-baa8-89007295f104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:40.728144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:40.758762 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc1fcb7-4a06-4ca0-b860-b8879b97bddf None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:40.775007 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 540/1091] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:16:40.792920 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a06a7-1c95-4710-9881-b1d03669a80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:40.793602 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a06a7-1c95-4710-9881-b1d03669a80c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:40.793902 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a06a7-1c95-4710-9881-b1d03669a80c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:40.794176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a06a7-1c95-4710-9881-b1d03669a80c None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:40.794965 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a06a7-1c95-4710-9881-b1d03669a80c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:40.795285 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a06a7-1c95-4710-9881-b1d03669a80c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:40.796730 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a06a7-1c95-4710-9881-b1d03669a80c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:40.797751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5478eca-5ff3-4a1c-baa8-89007295f104 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:40.810215 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 552/1092] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:16:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:40.815903 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a06a7-1c95-4710-9881-b1d03669a80c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:40.843969 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:40.898134 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5478eca-5ff3-4a1c-baa8-89007295f104 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:40.915374 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a06a7-1c95-4710-9881-b1d03669a80c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:40.932775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81184888-05d1-4d30-85e4-2878c7258e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:40.933569 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81184888-05d1-4d30-85e4-2878c7258e86 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:40.933888 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81184888-05d1-4d30-85e4-2878c7258e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:40.934170 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81184888-05d1-4d30-85e4-2878c7258e86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:40.944114 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81184888-05d1-4d30-85e4-2878c7258e86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:16:40.944328 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:40.949749 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a13f0-dde7-42c3-a823-087dc6821957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:40.950465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a13f0-dde7-42c3-a823-087dc6821957 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:40.950987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a13f0-dde7-42c3-a823-087dc6821957 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:40.951269 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a13f0-dde7-42c3-a823-087dc6821957 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:40.952097 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a13f0-dde7-42c3-a823-087dc6821957 None admin] RBAC: Adding query filter params (role.id=fca7293e63b14558bb206aaac4e5d1ca, scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b, user.id=b8751a4e92174d18b7b138392d2579a1) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:40.952194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-81184888-05d1-4d30-85e4-2878c7258e86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:16:40.952194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:40.952194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:40.952194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:40.952194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:40.952194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:40.952194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:40.952194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:40.952194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:40.952194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:40.952194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:40.952194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:40.952194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:16:40.952194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:16:40.952194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:16:40.952194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:16:40.953283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:16:40.953283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:16:40.953283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:16:40.953283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:16:40.953283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:40.953283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:40.953283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:40.953283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:40.953283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:40.953283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:40.953283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:40.953283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:40.953283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:40.953283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:40.953283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:40.953283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:40.954743 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:40.954743 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:40.954743 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:40.954743 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:40.954743 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:16:40.954743 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:16:40.954743 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:16:40.954743 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:16:40.954743 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:16:40.954743 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:16:40.954743 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:16:40.954743 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:40.955443 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a13f0-dde7-42c3-a823-087dc6821957 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:40.955526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 541/1093] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:16:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:16:40.955596 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a13f0-dde7-42c3-a823-087dc6821957 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:40.960032 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81184888-05d1-4d30-85e4-2878c7258e86 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:40.961214 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 553/1094] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:16:40 2022] GET /identity/v3/role_assignments?user.id=b8751a4e92174d18b7b138392d2579a1&scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b&role.id=fca7293e63b14558bb206aaac4e5d1ca => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:41.077753 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81184888-05d1-4d30-85e4-2878c7258e86 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:41.080262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:41.113462 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed19a8-7f31-451c-81ef-eee274513179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:41.114167 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed19a8-7f31-451c-81ef-eee274513179 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:41.114456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed19a8-7f31-451c-81ef-eee274513179 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:41.114738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed19a8-7f31-451c-81ef-eee274513179 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:41.115990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed19a8-7f31-451c-81ef-eee274513179 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:41.116296 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed19a8-7f31-451c-81ef-eee274513179 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:41.117849 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed19a8-7f31-451c-81ef-eee274513179 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:41.126476 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 542/1095] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:16:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:41.133776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a13f0-dde7-42c3-a823-087dc6821957 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:41.239604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a13f0-dde7-42c3-a823-087dc6821957 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:41.276544 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aab162-8c21-44d4-b699-a00fc6f475a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:41.277354 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aab162-8c21-44d4-b699-a00fc6f475a4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:41.277670 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aab162-8c21-44d4-b699-a00fc6f475a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:41.277954 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aab162-8c21-44d4-b699-a00fc6f475a4 None admin] PATH_INFO: `/v3/users/98c171b0c6cb4717af16f8f8fc939a9d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:41.279161 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aab162-8c21-44d4-b699-a00fc6f475a4 None admin] RBAC: Authorizing `identity:get_user(user_id=98c171b0c6cb4717af16f8f8fc939a9d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:41.280613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aab162-8c21-44d4-b699-a00fc6f475a4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:41.282294 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 554/1096] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:16:41 2022] GET /identity/v3/users/98c171b0c6cb4717af16f8f8fc939a9d => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:41.289031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed19a8-7f31-451c-81ef-eee274513179 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:41.401774 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed19a8-7f31-451c-81ef-eee274513179 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:41.436868 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339c0df-f0e6-4add-84f6-eafc0728172e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:41.437758 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339c0df-f0e6-4add-84f6-eafc0728172e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:41.438050 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339c0df-f0e6-4add-84f6-eafc0728172e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:41.438331 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339c0df-f0e6-4add-84f6-eafc0728172e None admin] PATH_INFO: `/v3/projects/f322a75bee2e40dfb392011793ec5104` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:41.439531 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3339c0df-f0e6-4add-84f6-eafc0728172e None admin] RBAC: Authorizing `identity:get_project(project_id=f322a75bee2e40dfb392011793ec5104)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:41.440979 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3339c0df-f0e6-4add-84f6-eafc0728172e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:41.442722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 543/1097] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:16:41 2022] GET /identity/v3/projects/f322a75bee2e40dfb392011793ec5104 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:41.449409 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aab162-8c21-44d4-b699-a00fc6f475a4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:41.515589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:41.560773 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aab162-8c21-44d4-b699-a00fc6f475a4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:41.596496 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e656fca-6ade-446e-8e5c-b8e040e8bbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:41.597390 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e656fca-6ade-446e-8e5c-b8e040e8bbc0 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:41.597703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e656fca-6ade-446e-8e5c-b8e040e8bbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:41.597987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e656fca-6ade-446e-8e5c-b8e040e8bbc0 None admin] PATH_INFO: `/v3/projects/f322a75bee2e40dfb392011793ec5104/users/98c171b0c6cb4717af16f8f8fc939a9d/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:41.600183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e656fca-6ade-446e-8e5c-b8e040e8bbc0 None admin] RBAC: Authorizing `identity:create_grant(project_id=f322a75bee2e40dfb392011793ec5104, user_id=98c171b0c6cb4717af16f8f8fc939a9d, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:41.601860 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e656fca-6ade-446e-8e5c-b8e040e8bbc0 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:41.616770 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 555/1098] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:16:41 2022] PUT /identity/v3/projects/f322a75bee2e40dfb392011793ec5104/users/98c171b0c6cb4717af16f8f8fc939a9d/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:16:43.654847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3e031-fcef-4b5b-b9f2-88c7ccd8474f None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:43.655382 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3e031-fcef-4b5b-b9f2-88c7ccd8474f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:43.655382 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3e031-fcef-4b5b-b9f2-88c7ccd8474f None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:43.656757 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 544/1099] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:43.662678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fe91c-87f0-4427-aac5-f8b4890bdb94 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:43.663149 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fe91c-87f0-4427-aac5-f8b4890bdb94 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:43.663349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fe91c-87f0-4427-aac5-f8b4890bdb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:43.680252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b38fe91c-87f0-4427-aac5-f8b4890bdb94 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:43.681444 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b38fe91c-87f0-4427-aac5-f8b4890bdb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:43.746700 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 556/1100] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:43.772174 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e3e031-fcef-4b5b-b9f2-88c7ccd8474f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:43.818086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:43.886361 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e3e031-fcef-4b5b-b9f2-88c7ccd8474f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:43.918721 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d900a29-d8d0-448e-b871-e6c0d7d43af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:43.919439 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d900a29-d8d0-448e-b871-e6c0d7d43af6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:43.919726 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d900a29-d8d0-448e-b871-e6c0d7d43af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:43.920007 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d900a29-d8d0-448e-b871-e6c0d7d43af6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:43.931759 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d900a29-d8d0-448e-b871-e6c0d7d43af6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:16:43.941886 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-1d900a29-d8d0-448e-b871-e6c0d7d43af6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:16:43.941886 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:16:43.941886 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:16:43.941886 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:16:43.941886 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:16:43.941886 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:16:43.941886 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:16:43.941886 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:16:43.941886 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:16:43.941886 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:16:43.941886 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:16:43.941886 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:16:43.941886 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:16:43.941886 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:16:43.941886 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:16:43.941886 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:16:43.943030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:16:43.943030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:16:43.943030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:16:43.943030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:16:43.943030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:16:43.943030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:16:43.943030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:16:43.943030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:16:43.943030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:16:43.943030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:16:43.943030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:16:43.943030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:16:43.943030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:16:43.943030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:16:43.943030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:16:43.943030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:16:43.944646 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:16:43.944646 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:16:43.944646 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:16:43.944646 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:16:43.944646 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:16:43.944646 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:16:43.944646 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:16:43.944646 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:16:43.944646 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:16:43.944646 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:16:43.944646 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:16:43.944646 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:16:43.944646 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 545/1101] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:16:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:16:43.949947 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38fe91c-87f0-4427-aac5-f8b4890bdb94 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:44.003960 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:44.050284 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38fe91c-87f0-4427-aac5-f8b4890bdb94 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:44.086259 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6dc9b-a64b-4be1-aef8-b40334aee0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:44.086927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6dc9b-a64b-4be1-aef8-b40334aee0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:44.087215 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6dc9b-a64b-4be1-aef8-b40334aee0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:44.087499 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6dc9b-a64b-4be1-aef8-b40334aee0b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:44.088248 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6dc9b-a64b-4be1-aef8-b40334aee0b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:44.088562 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6dc9b-a64b-4be1-aef8-b40334aee0b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:44.090028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6dc9b-a64b-4be1-aef8-b40334aee0b4 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:44.098037 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 557/1102] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:16:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:44.104904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d900a29-d8d0-448e-b871-e6c0d7d43af6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:44.197991 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:44.205315 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d900a29-d8d0-448e-b871-e6c0d7d43af6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:44.238076 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44512d-88d5-4c1e-ad6b-517568916992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:44.238950 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44512d-88d5-4c1e-ad6b-517568916992 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:44.239229 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44512d-88d5-4c1e-ad6b-517568916992 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:44.239501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44512d-88d5-4c1e-ad6b-517568916992 None admin] PATH_INFO: `/v3/users/98c171b0c6cb4717af16f8f8fc939a9d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:44.240901 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44512d-88d5-4c1e-ad6b-517568916992 None admin] RBAC: Authorizing `identity:get_user(user_id=98c171b0c6cb4717af16f8f8fc939a9d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:44.242352 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44512d-88d5-4c1e-ad6b-517568916992 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:44.243963 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 546/1103] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:16:44 2022] GET /identity/v3/users/98c171b0c6cb4717af16f8f8fc939a9d => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:44.250325 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6dc9b-a64b-4be1-aef8-b40334aee0b4 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:44.348423 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6dc9b-a64b-4be1-aef8-b40334aee0b4 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:44.363763 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:44.380736 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebca2d-cbad-4280-a08f-2d8b99ca6de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:44.381916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebca2d-cbad-4280-a08f-2d8b99ca6de2 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:44.382201 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebca2d-cbad-4280-a08f-2d8b99ca6de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:44.382481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebca2d-cbad-4280-a08f-2d8b99ca6de2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:44.452421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebca2d-cbad-4280-a08f-2d8b99ca6de2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:44.453920 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebca2d-cbad-4280-a08f-2d8b99ca6de2 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:44.559076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 558/1104] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:16:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 18:16:44.565174 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44512d-88d5-4c1e-ad6b-517568916992 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:44.667908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44512d-88d5-4c1e-ad6b-517568916992 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:44.701305 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95418fc5-3e2a-4c1c-89b4-93658fe461b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:44.702166 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95418fc5-3e2a-4c1c-89b4-93658fe461b5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:44.702492 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95418fc5-3e2a-4c1c-89b4-93658fe461b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:44.702713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95418fc5-3e2a-4c1c-89b4-93658fe461b5 None admin] PATH_INFO: `/v3/projects/f322a75bee2e40dfb392011793ec5104` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:44.703885 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95418fc5-3e2a-4c1c-89b4-93658fe461b5 None admin] RBAC: Authorizing `identity:get_project(project_id=f322a75bee2e40dfb392011793ec5104)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:44.705305 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95418fc5-3e2a-4c1c-89b4-93658fe461b5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:44.707010 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 547/1105] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:16:44 2022] GET /identity/v3/projects/f322a75bee2e40dfb392011793ec5104 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:44.713160 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebca2d-cbad-4280-a08f-2d8b99ca6de2 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:44.730014 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:44.810763 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebca2d-cbad-4280-a08f-2d8b99ca6de2 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:44.843884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b732c25-4b5b-45e1-83c6-2f253ae04d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:44.844526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b732c25-4b5b-45e1-83c6-2f253ae04d87 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:44.844813 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b732c25-4b5b-45e1-83c6-2f253ae04d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:44.845097 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b732c25-4b5b-45e1-83c6-2f253ae04d87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:44.845950 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b732c25-4b5b-45e1-83c6-2f253ae04d87 None admin] RBAC: Adding query filter params (role.id=0ef3687af20a443281e3cfdceca3597d, scope.project.id=f322a75bee2e40dfb392011793ec5104, user.id=98c171b0c6cb4717af16f8f8fc939a9d) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:44.846264 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b732c25-4b5b-45e1-83c6-2f253ae04d87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:44.847694 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b732c25-4b5b-45e1-83c6-2f253ae04d87 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:44.853539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 559/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:16:44 2022] GET /identity/v3/role_assignments?user.id=98c171b0c6cb4717af16f8f8fc939a9d&scope.project.id=f322a75bee2e40dfb392011793ec5104&role.id=0ef3687af20a443281e3cfdceca3597d => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:16:46.927016 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6675a52-e491-45cf-bfac-4987425c551a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:46.927550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6675a52-e491-45cf-bfac-4987425c551a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:46.927550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6675a52-e491-45cf-bfac-4987425c551a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:46.928942 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 548/1107] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:46.934846 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b50cf-9dd8-4504-93da-d484271c15dc None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:46.935268 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b50cf-9dd8-4504-93da-d484271c15dc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:46.935416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b50cf-9dd8-4504-93da-d484271c15dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:46.952409 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-686b50cf-9dd8-4504-93da-d484271c15dc None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:46.953639 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-686b50cf-9dd8-4504-93da-d484271c15dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:47.023680 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 560/1108] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:47.048697 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6675a52-e491-45cf-bfac-4987425c551a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:47.079802 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:47.148747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6675a52-e491-45cf-bfac-4987425c551a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:47.182192 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31966e6-0335-471e-8b58-10283ec93157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:47.182902 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31966e6-0335-471e-8b58-10283ec93157 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:47.183189 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31966e6-0335-471e-8b58-10283ec93157 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:47.183469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31966e6-0335-471e-8b58-10283ec93157 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:47.184631 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31966e6-0335-471e-8b58-10283ec93157 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:47.186109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31966e6-0335-471e-8b58-10283ec93157 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:47.187725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 549/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:47.194217 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b50cf-9dd8-4504-93da-d484271c15dc None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:47.205715 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:47.296294 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b50cf-9dd8-4504-93da-d484271c15dc None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:47.334990 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4537e7-8305-4b7a-8a9d-70ad67604027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:47.336714 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4537e7-8305-4b7a-8a9d-70ad67604027 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:47.336998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4537e7-8305-4b7a-8a9d-70ad67604027 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:47.337283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4537e7-8305-4b7a-8a9d-70ad67604027 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:47.338191 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4537e7-8305-4b7a-8a9d-70ad67604027 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:47.340493 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4537e7-8305-4b7a-8a9d-70ad67604027 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:47.362152 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 561/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 30 18:16:47 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:16:49.403707 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fa9a8-803e-484f-aea3-2dc9f19dd20f None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:49.404222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fa9a8-803e-484f-aea3-2dc9f19dd20f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:49.404222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fa9a8-803e-484f-aea3-2dc9f19dd20f None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:49.405525 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 550/1111] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:49.410712 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5290b-a3e3-4ad3-9cc5-d49a67fb75da None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:49.411126 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5290b-a3e3-4ad3-9cc5-d49a67fb75da None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:49.411290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5290b-a3e3-4ad3-9cc5-d49a67fb75da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:49.429371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cfd5290b-a3e3-4ad3-9cc5-d49a67fb75da None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:49.430620 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cfd5290b-a3e3-4ad3-9cc5-d49a67fb75da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:49.489409 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:49.498068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 562/1112] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:16:49.522878 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fa9a8-803e-484f-aea3-2dc9f19dd20f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:49.629312 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:49.635242 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fa9a8-803e-484f-aea3-2dc9f19dd20f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:49.673329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ede39-23bc-48dd-b77c-feb5b25a6119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:49.674783 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ede39-23bc-48dd-b77c-feb5b25a6119 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:49.675361 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ede39-23bc-48dd-b77c-feb5b25a6119 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:49.675521 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ede39-23bc-48dd-b77c-feb5b25a6119 None admin] PATH_INFO: `/v3/roles/d5f874a3d78f4ee0ade318e1bbebf7ec` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:49.677043 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ede39-23bc-48dd-b77c-feb5b25a6119 None admin] RBAC: Authorizing `identity:get_role(role_id=d5f874a3d78f4ee0ade318e1bbebf7ec)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:49.678600 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ede39-23bc-48dd-b77c-feb5b25a6119 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:49.679998 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 551/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:16:49 2022] GET /identity/v3/roles/d5f874a3d78f4ee0ade318e1bbebf7ec => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:49.685867 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5290b-a3e3-4ad3-9cc5-d49a67fb75da None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:49.782184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5290b-a3e3-4ad3-9cc5-d49a67fb75da None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:49.814738 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecafbde-4dd5-46c1-a0cb-731029f9b8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:49.815597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecafbde-4dd5-46c1-a0cb-731029f9b8ab None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:49.815878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecafbde-4dd5-46c1-a0cb-731029f9b8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:49.816153 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecafbde-4dd5-46c1-a0cb-731029f9b8ab None admin] PATH_INFO: `/v3/users/d8e8d4ee66a54f83a496e0152beec764` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:49.829451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecafbde-4dd5-46c1-a0cb-731029f9b8ab None admin] RBAC: Authorizing `identity:get_user(user_id=d8e8d4ee66a54f83a496e0152beec764)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:49.830920 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecafbde-4dd5-46c1-a0cb-731029f9b8ab None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:49.832567 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 563/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:16:49 2022] GET /identity/v3/users/d8e8d4ee66a54f83a496e0152beec764 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:49.838531 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ede39-23bc-48dd-b77c-feb5b25a6119 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:49.941933 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ede39-23bc-48dd-b77c-feb5b25a6119 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:49.975133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaafea5-1f04-48d2-b450-33668d73fec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:49.975776 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaafea5-1f04-48d2-b450-33668d73fec5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:49.976065 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaafea5-1f04-48d2-b450-33668d73fec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:49.976349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaafea5-1f04-48d2-b450-33668d73fec5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:50.010291 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:50.049874 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaafea5-1f04-48d2-b450-33668d73fec5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:50.051337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaafea5-1f04-48d2-b450-33668d73fec5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:50.149919 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 552/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:16:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 18:16:50.155962 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecafbde-4dd5-46c1-a0cb-731029f9b8ab None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:50.214726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:50.261836 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecafbde-4dd5-46c1-a0cb-731029f9b8ab None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:50.298260 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a5916-3187-4f31-9567-1074403212f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:50.299124 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a5916-3187-4f31-9567-1074403212f7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:50.299404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a5916-3187-4f31-9567-1074403212f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:50.299686 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a5916-3187-4f31-9567-1074403212f7 None admin] PATH_INFO: `/v3/projects/f322a75bee2e40dfb392011793ec5104` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:50.300877 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a5916-3187-4f31-9567-1074403212f7 None admin] RBAC: Authorizing `identity:get_project(project_id=f322a75bee2e40dfb392011793ec5104)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:50.302354 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a5916-3187-4f31-9567-1074403212f7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:50.304629 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 564/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:16:50 2022] GET /identity/v3/projects/f322a75bee2e40dfb392011793ec5104 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:50.310774 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaafea5-1f04-48d2-b450-33668d73fec5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:50.384625 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:50.421415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaafea5-1f04-48d2-b450-33668d73fec5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:50.458346 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d113c-32f4-4e41-8b4a-45072bd2d0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:50.459007 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d113c-32f4-4e41-8b4a-45072bd2d0ae None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:50.459322 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d113c-32f4-4e41-8b4a-45072bd2d0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:50.459598 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d113c-32f4-4e41-8b4a-45072bd2d0ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:50.460443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d113c-32f4-4e41-8b4a-45072bd2d0ae None admin] RBAC: Adding query filter params (role.id=d5f874a3d78f4ee0ade318e1bbebf7ec, scope.project.id=f322a75bee2e40dfb392011793ec5104, user.id=d8e8d4ee66a54f83a496e0152beec764) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:50.460753 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d113c-32f4-4e41-8b4a-45072bd2d0ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:50.462244 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d113c-32f4-4e41-8b4a-45072bd2d0ae None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:50.468307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 553/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:16:50 2022] GET /identity/v3/role_assignments?user.id=d8e8d4ee66a54f83a496e0152beec764&scope.project.id=f322a75bee2e40dfb392011793ec5104&role.id=d5f874a3d78f4ee0ade318e1bbebf7ec => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:52.531606 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcb160-fe9c-42a7-b003-780f2902cb9e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:52.532136 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcb160-fe9c-42a7-b003-780f2902cb9e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:52.532251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcb160-fe9c-42a7-b003-780f2902cb9e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:52.533525 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 565/1118] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:52.540221 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5201e-425f-4250-9579-89e9c63f52d6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:52.540664 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5201e-425f-4250-9579-89e9c63f52d6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:52.540767 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5201e-425f-4250-9579-89e9c63f52d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:52.558064 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a8d5201e-425f-4250-9579-89e9c63f52d6 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:52.559231 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a8d5201e-425f-4250-9579-89e9c63f52d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:52.623819 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 554/1119] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:52.646422 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcb160-fe9c-42a7-b003-780f2902cb9e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:52.755703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcb160-fe9c-42a7-b003-780f2902cb9e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:52.779534 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:52.792742 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a189e-f5c1-42c6-8023-e1967bb4f40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:52.793528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a189e-f5c1-42c6-8023-e1967bb4f40f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:52.793845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a189e-f5c1-42c6-8023-e1967bb4f40f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:52.794127 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a189e-f5c1-42c6-8023-e1967bb4f40f None admin] PATH_INFO: `/v3/roles/d5f874a3d78f4ee0ade318e1bbebf7ec` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:52.795597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a189e-f5c1-42c6-8023-e1967bb4f40f None admin] RBAC: Authorizing `identity:get_role(role_id=d5f874a3d78f4ee0ade318e1bbebf7ec)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:52.796991 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a189e-f5c1-42c6-8023-e1967bb4f40f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:52.798677 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 566/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:16:52 2022] GET /identity/v3/roles/d5f874a3d78f4ee0ade318e1bbebf7ec => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:52.805528 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5201e-425f-4250-9579-89e9c63f52d6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:52.864408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:52.905679 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5201e-425f-4250-9579-89e9c63f52d6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:52.940256 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bdcb5d-3598-49e4-a40c-ab3e6ed57c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:52.941113 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bdcb5d-3598-49e4-a40c-ab3e6ed57c6b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:52.941394 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bdcb5d-3598-49e4-a40c-ab3e6ed57c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:52.941697 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bdcb5d-3598-49e4-a40c-ab3e6ed57c6b None admin] PATH_INFO: `/v3/users/d8e8d4ee66a54f83a496e0152beec764` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:52.942882 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bdcb5d-3598-49e4-a40c-ab3e6ed57c6b None admin] RBAC: Authorizing `identity:get_user(user_id=d8e8d4ee66a54f83a496e0152beec764)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:52.944402 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bdcb5d-3598-49e4-a40c-ab3e6ed57c6b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:52.946088 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 555/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:16:52 2022] GET /identity/v3/users/d8e8d4ee66a54f83a496e0152beec764 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:52.952223 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a189e-f5c1-42c6-8023-e1967bb4f40f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:53.056034 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a189e-f5c1-42c6-8023-e1967bb4f40f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:53.091679 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c6977e-bf06-44f6-ad40-e5dd209a4e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:53.092537 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6977e-bf06-44f6-ad40-e5dd209a4e80 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:53.092821 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6977e-bf06-44f6-ad40-e5dd209a4e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:53.093100 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6977e-bf06-44f6-ad40-e5dd209a4e80 None admin] PATH_INFO: `/v3/projects/f322a75bee2e40dfb392011793ec5104` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:53.094302 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c6977e-bf06-44f6-ad40-e5dd209a4e80 None admin] RBAC: Authorizing `identity:get_project(project_id=f322a75bee2e40dfb392011793ec5104)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:53.095747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c6977e-bf06-44f6-ad40-e5dd209a4e80 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:53.097473 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 567/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:16:52 2022] GET /identity/v3/projects/f322a75bee2e40dfb392011793ec5104 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:53.105422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bdcb5d-3598-49e4-a40c-ab3e6ed57c6b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:53.211834 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bdcb5d-3598-49e4-a40c-ab3e6ed57c6b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:53.240094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:53.249136 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e2c6c-db2e-4399-bb78-b826cfa4001f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:53.250075 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e2c6c-db2e-4399-bb78-b826cfa4001f None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:53.250359 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e2c6c-db2e-4399-bb78-b826cfa4001f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:53.250636 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e2c6c-db2e-4399-bb78-b826cfa4001f None admin] PATH_INFO: `/v3/projects/f322a75bee2e40dfb392011793ec5104/users/d8e8d4ee66a54f83a496e0152beec764/roles/d5f874a3d78f4ee0ade318e1bbebf7ec` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:53.252866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e2c6c-db2e-4399-bb78-b826cfa4001f None admin] RBAC: Authorizing `identity:create_grant(project_id=f322a75bee2e40dfb392011793ec5104, user_id=d8e8d4ee66a54f83a496e0152beec764, role_id=d5f874a3d78f4ee0ade318e1bbebf7ec)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:53.254352 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e2c6c-db2e-4399-bb78-b826cfa4001f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:53.269257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 556/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:16:53 2022] PUT /identity/v3/projects/f322a75bee2e40dfb392011793ec5104/users/d8e8d4ee66a54f83a496e0152beec764/roles/d5f874a3d78f4ee0ade318e1bbebf7ec => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:16:55.315345 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72336308-1415-441b-ae3e-d86f3790a2df None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:55.315848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72336308-1415-441b-ae3e-d86f3790a2df None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:55.315848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72336308-1415-441b-ae3e-d86f3790a2df None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:55.317280 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 568/1124] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:55.322586 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43f727-4c4b-4dce-9f3a-3cd4ba7277a7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:55.323059 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43f727-4c4b-4dce-9f3a-3cd4ba7277a7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:55.323149 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43f727-4c4b-4dce-9f3a-3cd4ba7277a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:55.340404 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3d43f727-4c4b-4dce-9f3a-3cd4ba7277a7 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:55.341640 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3d43f727-4c4b-4dce-9f3a-3cd4ba7277a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:55.408994 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 557/1125] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:55.433097 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72336308-1415-441b-ae3e-d86f3790a2df None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:55.490878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:55.532395 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72336308-1415-441b-ae3e-d86f3790a2df None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:55.566554 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dbdfb6-fbda-48a2-b41d-bd11c1e1a822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:55.567327 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbdfb6-fbda-48a2-b41d-bd11c1e1a822 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:55.567601 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbdfb6-fbda-48a2-b41d-bd11c1e1a822 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:55.567877 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbdfb6-fbda-48a2-b41d-bd11c1e1a822 None admin] PATH_INFO: `/v3/roles/d5f874a3d78f4ee0ade318e1bbebf7ec` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:55.570162 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dbdfb6-fbda-48a2-b41d-bd11c1e1a822 None admin] RBAC: Authorizing `identity:get_role(role_id=d5f874a3d78f4ee0ade318e1bbebf7ec)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:55.571535 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dbdfb6-fbda-48a2-b41d-bd11c1e1a822 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:55.572905 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 569/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:16:55 2022] GET /identity/v3/roles/d5f874a3d78f4ee0ade318e1bbebf7ec => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:55.578871 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43f727-4c4b-4dce-9f3a-3cd4ba7277a7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:55.677205 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43f727-4c4b-4dce-9f3a-3cd4ba7277a7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:55.695275 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:55.713668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8daa40-7e0c-4606-b777-e9b540302739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:55.714542 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8daa40-7e0c-4606-b777-e9b540302739 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:55.714821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8daa40-7e0c-4606-b777-e9b540302739 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:55.715093 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8daa40-7e0c-4606-b777-e9b540302739 None admin] PATH_INFO: `/v3/users/d8e8d4ee66a54f83a496e0152beec764` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:55.716283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8daa40-7e0c-4606-b777-e9b540302739 None admin] RBAC: Authorizing `identity:get_user(user_id=d8e8d4ee66a54f83a496e0152beec764)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:55.717749 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8daa40-7e0c-4606-b777-e9b540302739 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:55.719405 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 558/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:16:55 2022] GET /identity/v3/users/d8e8d4ee66a54f83a496e0152beec764 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:55.725504 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dbdfb6-fbda-48a2-b41d-bd11c1e1a822 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:55.824559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dbdfb6-fbda-48a2-b41d-bd11c1e1a822 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:55.849885 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:55.859347 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbcb30d-7a94-4060-8982-a7023b15f952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:55.859988 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcb30d-7a94-4060-8982-a7023b15f952 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:55.860271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcb30d-7a94-4060-8982-a7023b15f952 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:55.860548 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcb30d-7a94-4060-8982-a7023b15f952 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:55.925332 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbcb30d-7a94-4060-8982-a7023b15f952 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:55.926802 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbcb30d-7a94-4060-8982-a7023b15f952 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:56.023119 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 570/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:16:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 18:16:56.028989 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8daa40-7e0c-4606-b777-e9b540302739 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:56.129103 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8daa40-7e0c-4606-b777-e9b540302739 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:56.164513 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4cc7fb-5d58-4c22-a307-4d6fd5ec88ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:56.165368 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cc7fb-5d58-4c22-a307-4d6fd5ec88ad None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:56.165689 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cc7fb-5d58-4c22-a307-4d6fd5ec88ad None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:56.165975 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cc7fb-5d58-4c22-a307-4d6fd5ec88ad None admin] PATH_INFO: `/v3/projects/f322a75bee2e40dfb392011793ec5104` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:56.167268 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4cc7fb-5d58-4c22-a307-4d6fd5ec88ad None admin] RBAC: Authorizing `identity:get_project(project_id=f322a75bee2e40dfb392011793ec5104)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:56.168641 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4cc7fb-5d58-4c22-a307-4d6fd5ec88ad None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:56.170403 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 559/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:16:56 2022] GET /identity/v3/projects/f322a75bee2e40dfb392011793ec5104 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:56.176819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbcb30d-7a94-4060-8982-a7023b15f952 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:56.203793 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:56.280143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbcb30d-7a94-4060-8982-a7023b15f952 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:56.313807 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857d666-ef54-4da1-974c-72bbe087c2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:56.314460 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857d666-ef54-4da1-974c-72bbe087c2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:56.314748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857d666-ef54-4da1-974c-72bbe087c2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:56.315028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857d666-ef54-4da1-974c-72bbe087c2e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:56.315859 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857d666-ef54-4da1-974c-72bbe087c2e9 None admin] RBAC: Adding query filter params (role.id=d5f874a3d78f4ee0ade318e1bbebf7ec, scope.project.id=f322a75bee2e40dfb392011793ec5104, user.id=d8e8d4ee66a54f83a496e0152beec764) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:16:56.316393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857d666-ef54-4da1-974c-72bbe087c2e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:56.317630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857d666-ef54-4da1-974c-72bbe087c2e9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:56.324277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 571/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:16:56 2022] GET /identity/v3/role_assignments?user.id=d8e8d4ee66a54f83a496e0152beec764&scope.project.id=f322a75bee2e40dfb392011793ec5104&role.id=d5f874a3d78f4ee0ade318e1bbebf7ec => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:58.450402 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f937e9-d3a0-42db-983b-b4d2061995c8 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:58.450975 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f937e9-d3a0-42db-983b-b4d2061995c8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:58.451075 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f937e9-d3a0-42db-983b-b4d2061995c8 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:58.452452 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 560/1131] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:16:58.457836 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb14334-3d13-46b9-bf53-5016ff3f0b5a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:58.458291 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb14334-3d13-46b9-bf53-5016ff3f0b5a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:58.458418 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb14334-3d13-46b9-bf53-5016ff3f0b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:58.477818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-afb14334-3d13-46b9-bf53-5016ff3f0b5a None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:16:58.479031 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-afb14334-3d13-46b9-bf53-5016ff3f0b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:58.548186 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 572/1132] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:16:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:16:58.573006 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f937e9-d3a0-42db-983b-b4d2061995c8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:58.632670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:58.680912 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f937e9-d3a0-42db-983b-b4d2061995c8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:58.717088 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357c62a-34ab-4ca9-b2b1-32805e493d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:58.717825 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357c62a-34ab-4ca9-b2b1-32805e493d2a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:58.718106 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357c62a-34ab-4ca9-b2b1-32805e493d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:58.718384 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357c62a-34ab-4ca9-b2b1-32805e493d2a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:58.719628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6357c62a-34ab-4ca9-b2b1-32805e493d2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:58.721292 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6357c62a-34ab-4ca9-b2b1-32805e493d2a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:58.723140 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 561/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:16:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:16:58.730923 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb14334-3d13-46b9-bf53-5016ff3f0b5a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:58.753262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:58.832978 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb14334-3d13-46b9-bf53-5016ff3f0b5a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:58.869409 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8308c-f069-49b6-ad31-4f574da85183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:58.874168 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8308c-f069-49b6-ad31-4f574da85183 None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:58.874168 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8308c-f069-49b6-ad31-4f574da85183 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:58.874168 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8308c-f069-49b6-ad31-4f574da85183 None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:58.874168 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8308c-f069-49b6-ad31-4f574da85183 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:58.874168 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8308c-f069-49b6-ad31-4f574da85183 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:58.899841 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 573/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 30 18:16:58 2022] POST /identity/v3/projects => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:17:01.014590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371de19-4d79-4c5d-8fa9-9855fb15f5c8 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:01.015160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371de19-4d79-4c5d-8fa9-9855fb15f5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:01.015160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371de19-4d79-4c5d-8fa9-9855fb15f5c8 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:01.016532 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 562/1135] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:17:01.022233 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8271f7-966a-4a38-8847-7bddd11f731f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:01.022773 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8271f7-966a-4a38-8847-7bddd11f731f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:01.022981 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8271f7-966a-4a38-8847-7bddd11f731f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:01.040689 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ca8271f7-966a-4a38-8847-7bddd11f731f None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:17:01.041918 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ca8271f7-966a-4a38-8847-7bddd11f731f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:17:01.128075 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 574/1136] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:17:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:17:01.154215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371de19-4d79-4c5d-8fa9-9855fb15f5c8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:01.262083 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371de19-4d79-4c5d-8fa9-9855fb15f5c8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:01.285465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:01.298334 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac3392-1824-4c3d-b096-f9aeaf04a9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:01.299151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac3392-1824-4c3d-b096-f9aeaf04a9ae None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:01.299442 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac3392-1824-4c3d-b096-f9aeaf04a9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:01.299778 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac3392-1824-4c3d-b096-f9aeaf04a9ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:01.301029 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac3392-1824-4c3d-b096-f9aeaf04a9ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:01.302690 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac3392-1824-4c3d-b096-f9aeaf04a9ae None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:01.304375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 563/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:17:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:01.310804 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8271f7-966a-4a38-8847-7bddd11f731f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:01.313134 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:01.418218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8271f7-966a-4a38-8847-7bddd11f731f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:01.453590 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0d252-28e7-4461-9f44-d75e2761944a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:01.454282 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0d252-28e7-4461-9f44-d75e2761944a None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:01.454572 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0d252-28e7-4461-9f44-d75e2761944a None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:01.454854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0d252-28e7-4461-9f44-d75e2761944a None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:01.455574 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0d252-28e7-4461-9f44-d75e2761944a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:01.457016 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0d252-28e7-4461-9f44-d75e2761944a None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:01.477203 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 575/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 30 18:17:01 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:17:03.587420 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8705a7-b7c0-4e75-a8dd-e10b8c8248e0 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:03.588008 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8705a7-b7c0-4e75-a8dd-e10b8c8248e0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:03.588113 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8705a7-b7c0-4e75-a8dd-e10b8c8248e0 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:03.589731 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 564/1139] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:17:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:17:03.596426 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33808d7b-7246-4743-a3aa-e925f90c9bda None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:03.596912 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33808d7b-7246-4743-a3aa-e925f90c9bda None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:03.597013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33808d7b-7246-4743-a3aa-e925f90c9bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:03.614004 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-33808d7b-7246-4743-a3aa-e925f90c9bda None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:17:03.615215 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-33808d7b-7246-4743-a3aa-e925f90c9bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:17:03.680617 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:03.695739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 576/1140] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:17:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:17:03.720037 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8705a7-b7c0-4e75-a8dd-e10b8c8248e0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:03.828380 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8705a7-b7c0-4e75-a8dd-e10b8c8248e0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:03.863014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c067f-3e3e-4f32-94bd-14f4f28fbc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:03.863719 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c067f-3e3e-4f32-94bd-14f4f28fbc2b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:03.864000 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c067f-3e3e-4f32-94bd-14f4f28fbc2b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:03.864277 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c067f-3e3e-4f32-94bd-14f4f28fbc2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:03.875795 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c067f-3e3e-4f32-94bd-14f4f28fbc2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:17:03.883500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-8e1c067f-3e3e-4f32-94bd-14f4f28fbc2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:17:03.883500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:17:03.883500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:17:03.883500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:17:03.883500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:17:03.883500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:17:03.883500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:17:03.883500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:17:03.883500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:17:03.883500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:17:03.883500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:17:03.883500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:17:03.883500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:17:03.883500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:17:03.883500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:17:03.883500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:17:03.884820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:17:03.884820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:17:03.884820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:17:03.884820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:17:03.884820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:17:03.884820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:17:03.884820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:17:03.884820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:17:03.884820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:17:03.884820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:17:03.884820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:17:03.884820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:17:03.884820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:17:03.884820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:17:03.884820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:17:03.884820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:17:03.885977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:17:03.885977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:17:03.885977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:17:03.885977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:17:03.885977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:17:03.885977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:17:03.885977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:17:03.885977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:17:03.885977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:17:03.885977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:17:03.885977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:17:03.885977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:17:03.885977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 565/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:17:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:17:03.890873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33808d7b-7246-4743-a3aa-e925f90c9bda None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:03.993943 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33808d7b-7246-4743-a3aa-e925f90c9bda None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:04.028935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d82ad-c6d8-4fdb-9cc1-5a197b3aecc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:04.029630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d82ad-c6d8-4fdb-9cc1-5a197b3aecc8 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:04.029916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d82ad-c6d8-4fdb-9cc1-5a197b3aecc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:04.030193 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d82ad-c6d8-4fdb-9cc1-5a197b3aecc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:04.030996 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d82ad-c6d8-4fdb-9cc1-5a197b3aecc8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:17:04.031363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d82ad-c6d8-4fdb-9cc1-5a197b3aecc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:04.032774 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d82ad-c6d8-4fdb-9cc1-5a197b3aecc8 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:04.042245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 577/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:17:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:04.048101 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c067f-3e3e-4f32-94bd-14f4f28fbc2b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:04.101555 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:04.156665 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c067f-3e3e-4f32-94bd-14f4f28fbc2b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:04.191834 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49346376-2f9d-429d-aea9-715ad5225351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:04.192704 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49346376-2f9d-429d-aea9-715ad5225351 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:04.192989 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49346376-2f9d-429d-aea9-715ad5225351 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:04.193268 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49346376-2f9d-429d-aea9-715ad5225351 None admin] PATH_INFO: `/v3/users/608707ef5acb4aea9b37734a3eefc017` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:04.208405 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49346376-2f9d-429d-aea9-715ad5225351 None admin] RBAC: Authorizing `identity:get_user(user_id=608707ef5acb4aea9b37734a3eefc017)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:04.210424 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49346376-2f9d-429d-aea9-715ad5225351 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:04.212184 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 566/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:17:04 2022] GET /identity/v3/users/608707ef5acb4aea9b37734a3eefc017 => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:04.218139 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d82ad-c6d8-4fdb-9cc1-5a197b3aecc8 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:04.273746 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:04.323784 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d82ad-c6d8-4fdb-9cc1-5a197b3aecc8 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:04.357880 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ee309-94a3-4ab6-97ff-5710e363f701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:04.358529 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ee309-94a3-4ab6-97ff-5710e363f701 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:04.358820 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ee309-94a3-4ab6-97ff-5710e363f701 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:04.359097 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ee309-94a3-4ab6-97ff-5710e363f701 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:04.435602 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ee309-94a3-4ab6-97ff-5710e363f701 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:04.436985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ee309-94a3-4ab6-97ff-5710e363f701 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:04.488802 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:04.541372 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 578/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:17:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 18:17:04.547643 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49346376-2f9d-429d-aea9-715ad5225351 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:04.656680 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49346376-2f9d-429d-aea9-715ad5225351 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:04.668007 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:04.696755 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a2c29-4e66-4b72-b3eb-714190e92908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:04.697532 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a2c29-4e66-4b72-b3eb-714190e92908 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:04.697850 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a2c29-4e66-4b72-b3eb-714190e92908 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:04.698128 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a2c29-4e66-4b72-b3eb-714190e92908 None admin] PATH_INFO: `/v3/projects/3d22011ab86e4f66b27cd232125a162d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:04.699377 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3a2c29-4e66-4b72-b3eb-714190e92908 None admin] RBAC: Authorizing `identity:get_project(project_id=3d22011ab86e4f66b27cd232125a162d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:04.700916 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3a2c29-4e66-4b72-b3eb-714190e92908 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:04.702716 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 567/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:17:04 2022] GET /identity/v3/projects/3d22011ab86e4f66b27cd232125a162d => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:04.709114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ee309-94a3-4ab6-97ff-5710e363f701 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:04.815474 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ee309-94a3-4ab6-97ff-5710e363f701 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:04.851318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b0931-acde-419d-90e7-c64229c628dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:04.851892 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b0931-acde-419d-90e7-c64229c628dd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:04.852182 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b0931-acde-419d-90e7-c64229c628dd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:04.852461 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b0931-acde-419d-90e7-c64229c628dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:04.853307 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b0931-acde-419d-90e7-c64229c628dd None admin] RBAC: Adding query filter params (role.id=0ef3687af20a443281e3cfdceca3597d, scope.project.id=3d22011ab86e4f66b27cd232125a162d, user.id=608707ef5acb4aea9b37734a3eefc017) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:17:04.853655 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b0931-acde-419d-90e7-c64229c628dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:04.855116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b0931-acde-419d-90e7-c64229c628dd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:04.860974 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 579/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:17:04 2022] GET /identity/v3/role_assignments?user.id=608707ef5acb4aea9b37734a3eefc017&scope.project.id=3d22011ab86e4f66b27cd232125a162d&role.id=0ef3687af20a443281e3cfdceca3597d => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:06.970597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb904367-d640-4228-89a0-0937732a5066 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:06.971129 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb904367-d640-4228-89a0-0937732a5066 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:06.971129 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb904367-d640-4228-89a0-0937732a5066 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:06.972521 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 568/1147] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:17:06.979224 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703396c-c62e-43a9-8a50-ec8361166b40 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:06.979671 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703396c-c62e-43a9-8a50-ec8361166b40 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:06.979785 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703396c-c62e-43a9-8a50-ec8361166b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:06.998136 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5703396c-c62e-43a9-8a50-ec8361166b40 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:17:06.999338 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5703396c-c62e-43a9-8a50-ec8361166b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:17:07.009554 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:07.072750 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 580/1148] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:17:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:17:07.096874 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb904367-d640-4228-89a0-0937732a5066 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:07.204728 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb904367-d640-4228-89a0-0937732a5066 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:07.245963 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7380b7ad-eba7-486f-96e4-c22d32d49c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:07.246670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380b7ad-eba7-486f-96e4-c22d32d49c1f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:07.246957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380b7ad-eba7-486f-96e4-c22d32d49c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:07.247236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380b7ad-eba7-486f-96e4-c22d32d49c1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:07.258225 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7380b7ad-eba7-486f-96e4-c22d32d49c1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:17:07.265843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-7380b7ad-eba7-486f-96e4-c22d32d49c1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:17:07.265843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:17:07.265843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:17:07.265843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:17:07.265843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:17:07.265843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:17:07.265843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:17:07.265843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:17:07.265843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:17:07.265843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:17:07.265843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:17:07.265843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:17:07.265843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:17:07.265843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:17:07.265843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:17:07.265843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:17:07.267069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:17:07.267069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:17:07.267069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:17:07.267069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:17:07.267069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:17:07.267069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:17:07.267069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:17:07.267069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:17:07.267069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:17:07.267069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:17:07.267069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:17:07.267069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:17:07.267069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:17:07.267069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:17:07.267069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:17:07.267069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:17:07.267968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:17:07.267968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:17:07.267968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:17:07.267968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:17:07.267968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:17:07.267968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:17:07.267968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:17:07.267968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:17:07.267968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:17:07.267968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:17:07.267968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:17:07.267968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:17:07.267968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 569/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:17:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:17:07.274380 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703396c-c62e-43a9-8a50-ec8361166b40 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:07.374119 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703396c-c62e-43a9-8a50-ec8361166b40 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:07.410005 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194dec2-dc58-4679-8afd-0c0a49a4dbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:07.410671 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194dec2-dc58-4679-8afd-0c0a49a4dbfb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:07.410965 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194dec2-dc58-4679-8afd-0c0a49a4dbfb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:07.411250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194dec2-dc58-4679-8afd-0c0a49a4dbfb None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:07.412002 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194dec2-dc58-4679-8afd-0c0a49a4dbfb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:17:07.412315 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194dec2-dc58-4679-8afd-0c0a49a4dbfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:07.413763 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194dec2-dc58-4679-8afd-0c0a49a4dbfb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:07.422291 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 581/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:17:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:07.429362 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7380b7ad-eba7-486f-96e4-c22d32d49c1f None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:07.466809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:07.561829 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7380b7ad-eba7-486f-96e4-c22d32d49c1f None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:07.599023 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab722e5d-1366-4064-864c-19a98fd74c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:07.599894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab722e5d-1366-4064-864c-19a98fd74c40 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:07.600742 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab722e5d-1366-4064-864c-19a98fd74c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:07.601619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab722e5d-1366-4064-864c-19a98fd74c40 None admin] PATH_INFO: `/v3/users/608707ef5acb4aea9b37734a3eefc017` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:07.602871 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab722e5d-1366-4064-864c-19a98fd74c40 None admin] RBAC: Authorizing `identity:get_user(user_id=608707ef5acb4aea9b37734a3eefc017)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:07.604328 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab722e5d-1366-4064-864c-19a98fd74c40 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:07.606042 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 570/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:17:07 2022] GET /identity/v3/users/608707ef5acb4aea9b37734a3eefc017 => generated 286 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:07.612262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194dec2-dc58-4679-8afd-0c0a49a4dbfb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:07.622096 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:07.723491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194dec2-dc58-4679-8afd-0c0a49a4dbfb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:07.757553 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee958f-a2df-4521-a775-a8435e75b233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:07.758440 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee958f-a2df-4521-a775-a8435e75b233 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:07.758729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee958f-a2df-4521-a775-a8435e75b233 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:07.759009 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee958f-a2df-4521-a775-a8435e75b233 None admin] PATH_INFO: `/v3/projects/3d22011ab86e4f66b27cd232125a162d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:07.760186 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee958f-a2df-4521-a775-a8435e75b233 None admin] RBAC: Authorizing `identity:get_project(project_id=3d22011ab86e4f66b27cd232125a162d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:07.761661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee958f-a2df-4521-a775-a8435e75b233 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:07.763373 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 582/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:17:07 2022] GET /identity/v3/projects/3d22011ab86e4f66b27cd232125a162d => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:07.769341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab722e5d-1366-4064-864c-19a98fd74c40 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:07.883234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab722e5d-1366-4064-864c-19a98fd74c40 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:07.885463 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:07.917720 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a61cca1-9850-4edc-ad4e-cccf732de089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:07.918599 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61cca1-9850-4edc-ad4e-cccf732de089 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:07.918883 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61cca1-9850-4edc-ad4e-cccf732de089 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:07.919160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61cca1-9850-4edc-ad4e-cccf732de089 None admin] PATH_INFO: `/v3/projects/3d22011ab86e4f66b27cd232125a162d/users/608707ef5acb4aea9b37734a3eefc017/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:07.921122 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a61cca1-9850-4edc-ad4e-cccf732de089 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d22011ab86e4f66b27cd232125a162d, user_id=608707ef5acb4aea9b37734a3eefc017, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:07.922608 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a61cca1-9850-4edc-ad4e-cccf732de089 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:07.935577 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 571/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:17:07 2022] PUT /identity/v3/projects/3d22011ab86e4f66b27cd232125a162d/users/608707ef5acb4aea9b37734a3eefc017/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:17:09.999338 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7aff7-a089-4486-a720-a56f363e3657 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:09.999873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7aff7-a089-4486-a720-a56f363e3657 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:09.999873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7aff7-a089-4486-a720-a56f363e3657 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:10.001267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 583/1154] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:17:10.006976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f97acc-ca6a-445e-b41b-f667196c7c2c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:10.007448 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f97acc-ca6a-445e-b41b-f667196c7c2c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:10.007541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f97acc-ca6a-445e-b41b-f667196c7c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:10.024318 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-51f97acc-ca6a-445e-b41b-f667196c7c2c None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:17:10.025545 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-51f97acc-ca6a-445e-b41b-f667196c7c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:17:10.099186 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 572/1155] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:17:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:17:10.123155 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7aff7-a089-4486-a720-a56f363e3657 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:10.210106 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:10.245330 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7aff7-a089-4486-a720-a56f363e3657 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:10.281964 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bda7f03-c1f9-4a13-b74c-f003b1f84258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:10.282662 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda7f03-c1f9-4a13-b74c-f003b1f84258 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:10.282953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda7f03-c1f9-4a13-b74c-f003b1f84258 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:10.283234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda7f03-c1f9-4a13-b74c-f003b1f84258 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:10.294535 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bda7f03-c1f9-4a13-b74c-f003b1f84258 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:17:10.302375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-0bda7f03-c1f9-4a13-b74c-f003b1f84258 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:17:10.302375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:17:10.302375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:17:10.302375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:17:10.302375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:17:10.302375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:17:10.302375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:17:10.302375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:17:10.302375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:17:10.302375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:17:10.302375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:17:10.302375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:17:10.302375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:17:10.302375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:17:10.302375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:17:10.302375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:17:10.303506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:17:10.303506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:17:10.303506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:17:10.303506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:17:10.303506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:17:10.303506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:17:10.303506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:17:10.303506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:17:10.303506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:17:10.303506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:17:10.303506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:17:10.303506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:17:10.303506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:17:10.303506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:17:10.303506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:17:10.303506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:17:10.304351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:17:10.304351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:17:10.304351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:17:10.304351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:17:10.304351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:17:10.304351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:17:10.304351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:17:10.304351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:17:10.304351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:17:10.304351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:17:10.304351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:17:10.304351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:17:10.304351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 584/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:17:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:17:10.309629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f97acc-ca6a-445e-b41b-f667196c7c2c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:10.404074 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:10.411324 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f97acc-ca6a-445e-b41b-f667196c7c2c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:10.446463 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128eba1-a38c-427d-86c9-161b595d4a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:10.447110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128eba1-a38c-427d-86c9-161b595d4a5b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:10.447388 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128eba1-a38c-427d-86c9-161b595d4a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:10.447662 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128eba1-a38c-427d-86c9-161b595d4a5b None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:10.448396 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a128eba1-a38c-427d-86c9-161b595d4a5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:17:10.448701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a128eba1-a38c-427d-86c9-161b595d4a5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:10.450104 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a128eba1-a38c-427d-86c9-161b595d4a5b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:10.457938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 573/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:17:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:10.464436 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bda7f03-c1f9-4a13-b74c-f003b1f84258 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:10.579274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bda7f03-c1f9-4a13-b74c-f003b1f84258 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:10.612554 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a835b8a-e786-4d75-a283-9467fb95f188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:10.613425 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a835b8a-e786-4d75-a283-9467fb95f188 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:10.613742 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a835b8a-e786-4d75-a283-9467fb95f188 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:10.614018 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a835b8a-e786-4d75-a283-9467fb95f188 None admin] PATH_INFO: `/v3/users/608707ef5acb4aea9b37734a3eefc017` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:10.615196 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a835b8a-e786-4d75-a283-9467fb95f188 None admin] RBAC: Authorizing `identity:get_user(user_id=608707ef5acb4aea9b37734a3eefc017)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:10.616697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a835b8a-e786-4d75-a283-9467fb95f188 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:10.618345 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 585/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:17:10 2022] GET /identity/v3/users/608707ef5acb4aea9b37734a3eefc017 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:10.624458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128eba1-a38c-427d-86c9-161b595d4a5b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:10.725388 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128eba1-a38c-427d-86c9-161b595d4a5b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:10.758157 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5efee-9b73-4674-bacd-19dd1723bfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:10.758797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5efee-9b73-4674-bacd-19dd1723bfe5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:10.759083 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5efee-9b73-4674-bacd-19dd1723bfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:10.759366 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5efee-9b73-4674-bacd-19dd1723bfe5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:10.783671 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:10.826351 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c5efee-9b73-4674-bacd-19dd1723bfe5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:10.827807 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c5efee-9b73-4674-bacd-19dd1723bfe5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:10.922455 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 574/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:17:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 18:17:10.929191 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a835b8a-e786-4d75-a283-9467fb95f188 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:10.934853 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:11.033600 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a835b8a-e786-4d75-a283-9467fb95f188 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:11.067825 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a4209-67ac-470b-acc7-173a35417c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:11.068685 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a4209-67ac-470b-acc7-173a35417c08 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:11.069913 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a4209-67ac-470b-acc7-173a35417c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:11.070203 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a4209-67ac-470b-acc7-173a35417c08 None admin] PATH_INFO: `/v3/projects/3d22011ab86e4f66b27cd232125a162d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:11.071393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a4209-67ac-470b-acc7-173a35417c08 None admin] RBAC: Authorizing `identity:get_project(project_id=3d22011ab86e4f66b27cd232125a162d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:11.072841 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a4209-67ac-470b-acc7-173a35417c08 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:11.074581 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 586/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:17:10 2022] GET /identity/v3/projects/3d22011ab86e4f66b27cd232125a162d => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:11.080722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5efee-9b73-4674-bacd-19dd1723bfe5 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:11.134758 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:11.178064 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5efee-9b73-4674-bacd-19dd1723bfe5 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:11.211394 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af72e0-0456-4866-aa1f-d0c0627dd8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:11.212029 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af72e0-0456-4866-aa1f-d0c0627dd8df None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:11.212312 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af72e0-0456-4866-aa1f-d0c0627dd8df None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:11.212588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af72e0-0456-4866-aa1f-d0c0627dd8df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:11.213409 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af72e0-0456-4866-aa1f-d0c0627dd8df None admin] RBAC: Adding query filter params (role.id=0ef3687af20a443281e3cfdceca3597d, scope.project.id=3d22011ab86e4f66b27cd232125a162d, user.id=608707ef5acb4aea9b37734a3eefc017) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:17:11.213748 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af72e0-0456-4866-aa1f-d0c0627dd8df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:11.215181 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af72e0-0456-4866-aa1f-d0c0627dd8df None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:11.220817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 575/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:17:11 2022] GET /identity/v3/role_assignments?user.id=608707ef5acb4aea9b37734a3eefc017&scope.project.id=3d22011ab86e4f66b27cd232125a162d&role.id=0ef3687af20a443281e3cfdceca3597d => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:13.304228 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0ac92-ed41-4260-b1b3-8a8fd3f7fce5 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:13.304731 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0ac92-ed41-4260-b1b3-8a8fd3f7fce5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:13.304731 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0ac92-ed41-4260-b1b3-8a8fd3f7fce5 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:13.306124 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 587/1162] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:17:13.311332 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0d4d8-ed33-4298-a315-83ff601773ea None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:13.311729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0d4d8-ed33-4298-a315-83ff601773ea None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:13.311895 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0d4d8-ed33-4298-a315-83ff601773ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:13.335315 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-18e0d4d8-ed33-4298-a315-83ff601773ea None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:17:13.336636 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-18e0d4d8-ed33-4298-a315-83ff601773ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:17:13.402309 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 576/1163] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:17:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:17:13.426165 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0ac92-ed41-4260-b1b3-8a8fd3f7fce5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:13.516793 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:13.560917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0ac92-ed41-4260-b1b3-8a8fd3f7fce5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:13.605541 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac4b54-57cc-48ee-b9fe-f1c857bf9327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:13.606372 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac4b54-57cc-48ee-b9fe-f1c857bf9327 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:13.606907 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac4b54-57cc-48ee-b9fe-f1c857bf9327 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:13.607326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac4b54-57cc-48ee-b9fe-f1c857bf9327 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:13.696356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac4b54-57cc-48ee-b9fe-f1c857bf9327 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:13.698212 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac4b54-57cc-48ee-b9fe-f1c857bf9327 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:13.792258 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:13.832028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 588/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:17:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 18:17:13.840314 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e0d4d8-ed33-4298-a315-83ff601773ea None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:13.906405 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:13.974807 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e0d4d8-ed33-4298-a315-83ff601773ea None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:14.025373 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e38aed-e453-47a7-a638-76f07ecbced2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:14.026226 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e38aed-e453-47a7-a638-76f07ecbced2 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:14.026509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e38aed-e453-47a7-a638-76f07ecbced2 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:14.028264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e38aed-e453-47a7-a638-76f07ecbced2 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:14.036285 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e38aed-e453-47a7-a638-76f07ecbced2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:14.038768 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e38aed-e453-47a7-a638-76f07ecbced2 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:14.055241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e8e38aed-e453-47a7-a638-76f07ecbced2 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 30 18:17:14.055241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:17:14.055241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 30 18:17:14.055241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 30 18:17:14.055241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:17:14.055241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:17:14.055241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:17:14.055241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:17:14.055241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 30 18:17:14.055241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:17:14.055241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:17:14.055241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:17:14.055241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:17:14.055241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:17:14.055241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:17:14.055241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:17:14.055241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:17:14.056509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:17:14.056509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:17:14.056509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:17:14.056509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:17:14.056509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:17:14.056509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:17:14.056509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:17:14.056509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:17:14.056509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:17:14.056509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:17:14.056509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:17:14.056509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:17:14.056509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:17:14.056509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:17:14.056509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:17:14.056509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:17:14.057372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:17:14.057372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:17:14.057372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:17:14.057372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:17:14.057372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:17:14.057372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:17:14.057372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:17:14.057372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:17:14.057372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:17:14.057372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:17:14.057372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 30 18:17:14.057372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:17:14.057372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 30 18:17:14.057372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:17:14.057372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 577/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 30 18:17:13 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:17:14.063409 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac4b54-57cc-48ee-b9fe-f1c857bf9327 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:14.186716 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac4b54-57cc-48ee-b9fe-f1c857bf9327 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:14.221916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a987cf-7699-477f-b17b-f273ea5220aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:14.222579 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a987cf-7699-477f-b17b-f273ea5220aa None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:14.222858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a987cf-7699-477f-b17b-f273ea5220aa None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:14.223136 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a987cf-7699-477f-b17b-f273ea5220aa None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:14.223856 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a987cf-7699-477f-b17b-f273ea5220aa None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:17:14.224162 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a987cf-7699-477f-b17b-f273ea5220aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:14.225625 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a987cf-7699-477f-b17b-f273ea5220aa None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:14.234283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 589/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 30 18:17:14 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:16.295256 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acfdae-90db-4ba7-b550-dcc197f445ed None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:16.295759 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acfdae-90db-4ba7-b550-dcc197f445ed None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:16.295759 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acfdae-90db-4ba7-b550-dcc197f445ed None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:16.297103 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 578/1167] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:17:16.302803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490d603-72b4-4ef6-9cba-e484b3031cfb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:16.303279 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490d603-72b4-4ef6-9cba-e484b3031cfb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:16.303865 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490d603-72b4-4ef6-9cba-e484b3031cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:16.321563 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5490d603-72b4-4ef6-9cba-e484b3031cfb None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:17:16.322781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5490d603-72b4-4ef6-9cba-e484b3031cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:17:16.352216 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:16.397906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 590/1168] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:17:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:17:16.421414 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1acfdae-90db-4ba7-b550-dcc197f445ed None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:16.531052 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1acfdae-90db-4ba7-b550-dcc197f445ed None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:16.542915 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:16.567448 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e78756-794c-461a-aa3f-5de296ebffba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:16.568031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e78756-794c-461a-aa3f-5de296ebffba None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:16.568316 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e78756-794c-461a-aa3f-5de296ebffba None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:16.568594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e78756-794c-461a-aa3f-5de296ebffba None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:16.570535 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e78756-794c-461a-aa3f-5de296ebffba None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:16.572892 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e78756-794c-461a-aa3f-5de296ebffba None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:16.588980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 579/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Thu Jun 30 18:17:16 2022] POST /identity/v3/domains => generated 267 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:17:18.679065 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8b00c-74b2-406b-9bb9-0a5675cb4d90 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:18.679631 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8b00c-74b2-406b-9bb9-0a5675cb4d90 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:18.679631 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8b00c-74b2-406b-9bb9-0a5675cb4d90 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:18.681545 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 591/1170] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:17:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:17:18.686905 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116c63f-ce02-46d6-9e57-c81875c83f3b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:18.687354 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116c63f-ce02-46d6-9e57-c81875c83f3b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:18.687485 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116c63f-ce02-46d6-9e57-c81875c83f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:18.704416 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6116c63f-ce02-46d6-9e57-c81875c83f3b None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:17:18.705725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6116c63f-ce02-46d6-9e57-c81875c83f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:17:18.780194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 580/1171] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:17:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:17:18.805836 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8b00c-74b2-406b-9bb9-0a5675cb4d90 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:18.916981 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8b00c-74b2-406b-9bb9-0a5675cb4d90 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:18.955044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e904de1-496b-4286-b269-5433e10db892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:18.955934 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e904de1-496b-4286-b269-5433e10db892 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:18.956233 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e904de1-496b-4286-b269-5433e10db892 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:18.956508 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e904de1-496b-4286-b269-5433e10db892 None admin] PATH_INFO: `/v3/domains/d518fe95b68842ff881a565e2ab8f58e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:18.967517 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e904de1-496b-4286-b269-5433e10db892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d518fe95b68842ff881a565e2ab8f58e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:18.968969 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e904de1-496b-4286-b269-5433e10db892 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:18.970796 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 592/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Thu Jun 30 18:17:18 2022] GET /identity/v3/domains/d518fe95b68842ff881a565e2ab8f58e => generated 267 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:18.977310 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116c63f-ce02-46d6-9e57-c81875c83f3b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:19.073424 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:19.076534 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116c63f-ce02-46d6-9e57-c81875c83f3b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:19.112311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5162686-354d-4037-afe3-0ef6fdba8a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:19.112965 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5162686-354d-4037-afe3-0ef6fdba8a79 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:19.113252 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5162686-354d-4037-afe3-0ef6fdba8a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:19.113531 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5162686-354d-4037-afe3-0ef6fdba8a79 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:19.114273 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5162686-354d-4037-afe3-0ef6fdba8a79 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:19.115716 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5162686-354d-4037-afe3-0ef6fdba8a79 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:19.140460 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 581/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 30 18:17:18 2022] POST /identity/v3/projects => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:17:21.255696 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c1183-1480-4231-bea4-6aa574204b42 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:21.256216 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c1183-1480-4231-bea4-6aa574204b42 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:21.256330 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c1183-1480-4231-bea4-6aa574204b42 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:21.257654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 593/1174] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:17:21.264454 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f168e-a708-4c8d-9156-475bad54a49d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:21.264974 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f168e-a708-4c8d-9156-475bad54a49d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:21.265070 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f168e-a708-4c8d-9156-475bad54a49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:21.283158 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f64f168e-a708-4c8d-9156-475bad54a49d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:17:21.284305 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f64f168e-a708-4c8d-9156-475bad54a49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:17:21.353698 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 582/1175] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:17:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:17:21.380494 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c1183-1480-4231-bea4-6aa574204b42 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:21.411429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:21.490890 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c1183-1480-4231-bea4-6aa574204b42 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:21.526887 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20249cb6-1692-4359-8a2c-cbecd9d4c384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:21.527850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20249cb6-1692-4359-8a2c-cbecd9d4c384 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:21.528191 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20249cb6-1692-4359-8a2c-cbecd9d4c384 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:21.528468 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20249cb6-1692-4359-8a2c-cbecd9d4c384 None admin] PATH_INFO: `/v3/domains/d518fe95b68842ff881a565e2ab8f58e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:21.529814 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20249cb6-1692-4359-8a2c-cbecd9d4c384 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d518fe95b68842ff881a565e2ab8f58e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:21.531265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20249cb6-1692-4359-8a2c-cbecd9d4c384 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:21.532893 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 594/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Thu Jun 30 18:17:21 2022] GET /identity/v3/domains/d518fe95b68842ff881a565e2ab8f58e => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:21.538815 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f168e-a708-4c8d-9156-475bad54a49d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:21.648548 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:21.667429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f168e-a708-4c8d-9156-475bad54a49d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:21.702465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af40e5bc-d04b-4de2-b534-43a48fcb339c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:21.703138 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40e5bc-d04b-4de2-b534-43a48fcb339c None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:21.703409 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40e5bc-d04b-4de2-b534-43a48fcb339c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:21.703686 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40e5bc-d04b-4de2-b534-43a48fcb339c None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:21.704407 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af40e5bc-d04b-4de2-b534-43a48fcb339c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:21.705865 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af40e5bc-d04b-4de2-b534-43a48fcb339c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:21.725442 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 583/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 30 18:17:21 2022] POST /identity/v3/users => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:17:23.823604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec71710-1555-44e1-a77c-089493f8b4ea None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:23.824136 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec71710-1555-44e1-a77c-089493f8b4ea None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:23.824234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec71710-1555-44e1-a77c-089493f8b4ea None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:23.825542 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 595/1178] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:17:23.830923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8523b3-61ec-43c8-85f9-9bf5bdeb69ad None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:23.831426 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8523b3-61ec-43c8-85f9-9bf5bdeb69ad None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:23.831562 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8523b3-61ec-43c8-85f9-9bf5bdeb69ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:23.848863 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ee8523b3-61ec-43c8-85f9-9bf5bdeb69ad None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:17:23.850125 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ee8523b3-61ec-43c8-85f9-9bf5bdeb69ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:17:23.918850 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 584/1179] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:17:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:17:23.943853 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec71710-1555-44e1-a77c-089493f8b4ea None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:24.041190 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:24.051962 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec71710-1555-44e1-a77c-089493f8b4ea None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:24.090861 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc015153-e434-44d0-ada6-fbe6e1f82d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:24.091565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc015153-e434-44d0-ada6-fbe6e1f82d03 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:24.091852 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc015153-e434-44d0-ada6-fbe6e1f82d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:24.092131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc015153-e434-44d0-ada6-fbe6e1f82d03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:24.103518 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc015153-e434-44d0-ada6-fbe6e1f82d03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:17:24.111242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-cc015153-e434-44d0-ada6-fbe6e1f82d03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:17:24.111242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:17:24.111242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:17:24.111242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:17:24.111242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:17:24.111242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:17:24.111242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:17:24.111242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:17:24.111242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:17:24.111242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:17:24.111242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:17:24.111242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:17:24.111242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:17:24.111242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:17:24.111242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:17:24.111242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:17:24.112456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:17:24.112456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:17:24.112456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:17:24.112456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:17:24.112456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:17:24.112456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:17:24.112456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:17:24.112456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:17:24.112456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:17:24.112456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:17:24.112456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:17:24.112456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:17:24.112456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:17:24.112456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:17:24.112456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:17:24.112456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:17:24.113523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:17:24.113523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:17:24.113523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:17:24.113523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:17:24.113523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:17:24.113523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:17:24.113523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:17:24.113523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:17:24.113523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:17:24.113523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:17:24.113523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:17:24.113523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:17:24.113523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 596/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:17:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:17:24.119105 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8523b3-61ec-43c8-85f9-9bf5bdeb69ad None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:24.189488 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:24.226236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8523b3-61ec-43c8-85f9-9bf5bdeb69ad None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:24.263311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfc19f-7cf3-4d9c-a190-5cbecf0cc9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:24.263985 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfc19f-7cf3-4d9c-a190-5cbecf0cc9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:24.264267 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfc19f-7cf3-4d9c-a190-5cbecf0cc9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:24.264542 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfc19f-7cf3-4d9c-a190-5cbecf0cc9d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:24.265285 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfc19f-7cf3-4d9c-a190-5cbecf0cc9d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:17:24.265614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfc19f-7cf3-4d9c-a190-5cbecf0cc9d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:24.266997 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfc19f-7cf3-4d9c-a190-5cbecf0cc9d4 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:24.276051 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 585/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:17:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:17:24.282163 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc015153-e434-44d0-ada6-fbe6e1f82d03 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:24.399504 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc015153-e434-44d0-ada6-fbe6e1f82d03 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:24.449279 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b78959c-5ff5-4e03-95dd-9bb703f2b745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:24.450182 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78959c-5ff5-4e03-95dd-9bb703f2b745 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:24.450460 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78959c-5ff5-4e03-95dd-9bb703f2b745 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:24.450736 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78959c-5ff5-4e03-95dd-9bb703f2b745 None admin] PATH_INFO: `/v3/users/77b615708855457e870683bda0894718` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:24.465607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b78959c-5ff5-4e03-95dd-9bb703f2b745 None admin] RBAC: Authorizing `identity:get_user(user_id=77b615708855457e870683bda0894718)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:24.467024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b78959c-5ff5-4e03-95dd-9bb703f2b745 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:24.468727 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 597/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:17:24 2022] GET /identity/v3/users/77b615708855457e870683bda0894718 => generated 311 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:24.474542 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfc19f-7cf3-4d9c-a190-5cbecf0cc9d4 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:24.580467 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfc19f-7cf3-4d9c-a190-5cbecf0cc9d4 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:24.611005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:24.617408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f81a0bd-4ce7-44f8-9e66-33e5cebe050d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:24.619344 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81a0bd-4ce7-44f8-9e66-33e5cebe050d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:24.619630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81a0bd-4ce7-44f8-9e66-33e5cebe050d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:24.619907 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81a0bd-4ce7-44f8-9e66-33e5cebe050d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:24.692802 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f81a0bd-4ce7-44f8-9e66-33e5cebe050d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:24.694224 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f81a0bd-4ce7-44f8-9e66-33e5cebe050d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:24.798140 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 586/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:17:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 18:17:24.804457 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b78959c-5ff5-4e03-95dd-9bb703f2b745 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:24.816814 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:24.906214 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b78959c-5ff5-4e03-95dd-9bb703f2b745 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:24.939895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f35d1-82f4-4043-b959-c1a4987f733b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:24.940748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f35d1-82f4-4043-b959-c1a4987f733b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:24.941031 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f35d1-82f4-4043-b959-c1a4987f733b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:24.941307 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f35d1-82f4-4043-b959-c1a4987f733b None admin] PATH_INFO: `/v3/projects/d1956deab85f40baae0279db7a93a3df` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:24.942505 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f35d1-82f4-4043-b959-c1a4987f733b None admin] RBAC: Authorizing `identity:get_project(project_id=d1956deab85f40baae0279db7a93a3df)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:24.943993 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f35d1-82f4-4043-b959-c1a4987f733b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:24.945694 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 598/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:17:24 2022] GET /identity/v3/projects/d1956deab85f40baae0279db7a93a3df => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:24.952181 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f81a0bd-4ce7-44f8-9e66-33e5cebe050d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:24.983524 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:25.056134 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f81a0bd-4ce7-44f8-9e66-33e5cebe050d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:25.091909 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84fbc6-6542-4cbb-9c0d-8024095014ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:25.092558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84fbc6-6542-4cbb-9c0d-8024095014ef None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:25.092833 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84fbc6-6542-4cbb-9c0d-8024095014ef None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:25.093109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84fbc6-6542-4cbb-9c0d-8024095014ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:25.093992 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84fbc6-6542-4cbb-9c0d-8024095014ef None admin] RBAC: Adding query filter params (role.id=0ef3687af20a443281e3cfdceca3597d, scope.project.id=d1956deab85f40baae0279db7a93a3df, user.id=77b615708855457e870683bda0894718) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:17:25.094304 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84fbc6-6542-4cbb-9c0d-8024095014ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:25.095743 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84fbc6-6542-4cbb-9c0d-8024095014ef None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:25.101347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 587/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:17:24 2022] GET /identity/v3/role_assignments?user.id=77b615708855457e870683bda0894718&scope.project.id=d1956deab85f40baae0279db7a93a3df&role.id=0ef3687af20a443281e3cfdceca3597d => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:27.156199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da32ea-5d80-45a4-83c4-f3064a207f0b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:27.156751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da32ea-5d80-45a4-83c4-f3064a207f0b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:27.156751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da32ea-5d80-45a4-83c4-f3064a207f0b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:27.158162 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 599/1186] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:17:27.163514 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15492798-47d3-42ea-87b7-ab06b5a677fe None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:27.163912 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15492798-47d3-42ea-87b7-ab06b5a677fe None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:27.164068 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15492798-47d3-42ea-87b7-ab06b5a677fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:27.181233 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-15492798-47d3-42ea-87b7-ab06b5a677fe None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:17:27.182430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-15492798-47d3-42ea-87b7-ab06b5a677fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:17:27.246588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 588/1187] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:17:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:17:27.268944 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da32ea-5d80-45a4-83c4-f3064a207f0b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:27.347420 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:27.370144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da32ea-5d80-45a4-83c4-f3064a207f0b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:27.402662 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69371800-345e-4cdd-ab90-2f5605a7058f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:27.403440 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69371800-345e-4cdd-ab90-2f5605a7058f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:27.403730 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69371800-345e-4cdd-ab90-2f5605a7058f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:27.404013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69371800-345e-4cdd-ab90-2f5605a7058f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:27.413994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69371800-345e-4cdd-ab90-2f5605a7058f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:17:27.422316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-69371800-345e-4cdd-ab90-2f5605a7058f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:17:27.422316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:17:27.422316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:17:27.422316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:17:27.422316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:17:27.422316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:17:27.422316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:17:27.422316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:17:27.422316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:17:27.422316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:17:27.422316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:17:27.422316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:17:27.422316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:17:27.422316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:17:27.422316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:17:27.422316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:17:27.423404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:17:27.423404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:17:27.423404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:17:27.423404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:17:27.423404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:17:27.423404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:17:27.423404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:17:27.423404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:17:27.423404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:17:27.423404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:17:27.423404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:17:27.423404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:17:27.423404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:17:27.423404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:17:27.423404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:17:27.423404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:17:27.424884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:17:27.424884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:17:27.424884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:17:27.424884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:17:27.424884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:17:27.424884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:17:27.424884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:17:27.424884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:17:27.424884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:17:27.424884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:17:27.424884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:17:27.424884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:17:27.424884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 600/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:17:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:17:27.431430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15492798-47d3-42ea-87b7-ab06b5a677fe None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:27.456032 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:27.531567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15492798-47d3-42ea-87b7-ab06b5a677fe None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:27.566493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e8340-83f6-46b4-b23d-84b2aaf5892c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:27.567154 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e8340-83f6-46b4-b23d-84b2aaf5892c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:27.567441 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e8340-83f6-46b4-b23d-84b2aaf5892c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:27.567720 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e8340-83f6-46b4-b23d-84b2aaf5892c None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:27.568460 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e8340-83f6-46b4-b23d-84b2aaf5892c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:17:27.568767 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e8340-83f6-46b4-b23d-84b2aaf5892c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:27.570175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e8340-83f6-46b4-b23d-84b2aaf5892c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:27.578153 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 589/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:17:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:27.584304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69371800-345e-4cdd-ab90-2f5605a7058f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:27.692959 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69371800-345e-4cdd-ab90-2f5605a7058f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:27.728816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f7721-3d7d-4bab-b905-671b000d4ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:27.729717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f7721-3d7d-4bab-b905-671b000d4ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:27.730009 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f7721-3d7d-4bab-b905-671b000d4ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:27.730292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f7721-3d7d-4bab-b905-671b000d4ae9 None admin] PATH_INFO: `/v3/users/77b615708855457e870683bda0894718` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:27.731509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f7721-3d7d-4bab-b905-671b000d4ae9 None admin] RBAC: Authorizing `identity:get_user(user_id=77b615708855457e870683bda0894718)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:27.732961 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f7721-3d7d-4bab-b905-671b000d4ae9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:27.734660 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 601/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:17:27 2022] GET /identity/v3/users/77b615708855457e870683bda0894718 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:27.740715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e8340-83f6-46b4-b23d-84b2aaf5892c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:27.828114 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:27.844494 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e8340-83f6-46b4-b23d-84b2aaf5892c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:27.879297 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb5364-8ecc-41f1-a1f9-52aa78360e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:27.880145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb5364-8ecc-41f1-a1f9-52aa78360e7a None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:27.880423 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb5364-8ecc-41f1-a1f9-52aa78360e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:27.880693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb5364-8ecc-41f1-a1f9-52aa78360e7a None admin] PATH_INFO: `/v3/projects/d1956deab85f40baae0279db7a93a3df` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:27.882013 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb5364-8ecc-41f1-a1f9-52aa78360e7a None admin] RBAC: Authorizing `identity:get_project(project_id=d1956deab85f40baae0279db7a93a3df)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:27.883484 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb5364-8ecc-41f1-a1f9-52aa78360e7a None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:27.885209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 590/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:17:27 2022] GET /identity/v3/projects/d1956deab85f40baae0279db7a93a3df => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:27.892692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f7721-3d7d-4bab-b905-671b000d4ae9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:27.903942 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:28.029246 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f7721-3d7d-4bab-b905-671b000d4ae9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:28.064695 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b7102-0460-4cdf-b99d-a95128736548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:28.065612 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b7102-0460-4cdf-b99d-a95128736548 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:28.065895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b7102-0460-4cdf-b99d-a95128736548 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:28.066175 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b7102-0460-4cdf-b99d-a95128736548 None admin] PATH_INFO: `/v3/projects/d1956deab85f40baae0279db7a93a3df/users/77b615708855457e870683bda0894718/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:28.068151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b7102-0460-4cdf-b99d-a95128736548 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1956deab85f40baae0279db7a93a3df, user_id=77b615708855457e870683bda0894718, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:28.069622 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b7102-0460-4cdf-b99d-a95128736548 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:28.084021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 602/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:17:27 2022] PUT /identity/v3/projects/d1956deab85f40baae0279db7a93a3df/users/77b615708855457e870683bda0894718/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:17:30.189811 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14e192-96ae-4d88-bfe7-c3087c01e13a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:30.190400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14e192-96ae-4d88-bfe7-c3087c01e13a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:30.190400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14e192-96ae-4d88-bfe7-c3087c01e13a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:30.191755 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 591/1193] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:17:30.197938 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596dc69-4c3f-4b09-b4f9-dcb8f6c96e0a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:30.198517 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596dc69-4c3f-4b09-b4f9-dcb8f6c96e0a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:30.198517 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596dc69-4c3f-4b09-b4f9-dcb8f6c96e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:30.219734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5596dc69-4c3f-4b09-b4f9-dcb8f6c96e0a None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:17:30.220942 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5596dc69-4c3f-4b09-b4f9-dcb8f6c96e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:17:30.287587 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:30.297340 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 603/1194] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:17:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:17:30.322255 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14e192-96ae-4d88-bfe7-c3087c01e13a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:30.427653 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14e192-96ae-4d88-bfe7-c3087c01e13a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:30.463362 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644db3a-f5c4-4d71-ab23-3d86a480edd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:30.464167 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644db3a-f5c4-4d71-ab23-3d86a480edd7 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:30.464453 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644db3a-f5c4-4d71-ab23-3d86a480edd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:30.464733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644db3a-f5c4-4d71-ab23-3d86a480edd7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:30.475763 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b644db3a-f5c4-4d71-ab23-3d86a480edd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:17:30.483898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-b644db3a-f5c4-4d71-ab23-3d86a480edd7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:17:30.483898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:17:30.483898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:17:30.483898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:17:30.483898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:17:30.483898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:17:30.483898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:17:30.483898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:17:30.483898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:17:30.483898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:17:30.483898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:17:30.483898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:17:30.483898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:17:30.483898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:17:30.483898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:17:30.483898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:17:30.485133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:17:30.485133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:17:30.485133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:17:30.485133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:17:30.485133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:17:30.485133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:17:30.485133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:17:30.485133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:17:30.485133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:17:30.485133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:17:30.485133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:17:30.485133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:17:30.485133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:17:30.485133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:17:30.485133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:17:30.485133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:17:30.486118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:17:30.486118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:17:30.486118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:17:30.486118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:17:30.486118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:17:30.486118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:17:30.486118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:17:30.486118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:17:30.486118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:17:30.486118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:17:30.486118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:17:30.486118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:17:30.486118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 592/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:17:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:17:30.491248 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596dc69-4c3f-4b09-b4f9-dcb8f6c96e0a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:30.595760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596dc69-4c3f-4b09-b4f9-dcb8f6c96e0a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:30.630188 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f630c-b928-420d-9729-c382107de3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:30.630860 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f630c-b928-420d-9729-c382107de3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:30.631144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f630c-b928-420d-9729-c382107de3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:30.631422 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f630c-b928-420d-9729-c382107de3d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:30.632171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f630c-b928-420d-9729-c382107de3d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:17:30.632481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f630c-b928-420d-9729-c382107de3d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:30.635170 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f630c-b928-420d-9729-c382107de3d5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:30.652209 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 604/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:17:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:30.658069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644db3a-f5c4-4d71-ab23-3d86a480edd7 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:30.674027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:30.762969 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644db3a-f5c4-4d71-ab23-3d86a480edd7 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:30.797990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d31d0e0-3d1c-4e6b-ad85-0956edc88ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:30.798855 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31d0e0-3d1c-4e6b-ad85-0956edc88ec3 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:30.799138 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31d0e0-3d1c-4e6b-ad85-0956edc88ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:30.799422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31d0e0-3d1c-4e6b-ad85-0956edc88ec3 None admin] PATH_INFO: `/v3/users/77b615708855457e870683bda0894718` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:30.800876 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d31d0e0-3d1c-4e6b-ad85-0956edc88ec3 None admin] RBAC: Authorizing `identity:get_user(user_id=77b615708855457e870683bda0894718)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:30.802357 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d31d0e0-3d1c-4e6b-ad85-0956edc88ec3 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:30.804308 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 593/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 30 18:17:30 2022] GET /identity/v3/users/77b615708855457e870683bda0894718 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:30.813284 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f630c-b928-420d-9729-c382107de3d5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:30.878637 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:30.922855 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f630c-b928-420d-9729-c382107de3d5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:30.957718 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a0b94-1c36-4fd9-b069-b3e256e05b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:30.958373 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a0b94-1c36-4fd9-b069-b3e256e05b97 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:30.958661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a0b94-1c36-4fd9-b069-b3e256e05b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:30.958945 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a0b94-1c36-4fd9-b069-b3e256e05b97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:31.028349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a0b94-1c36-4fd9-b069-b3e256e05b97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:31.029870 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a0b94-1c36-4fd9-b069-b3e256e05b97 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:31.089920 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:31.137790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 605/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:17:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 18:17:31.143983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d31d0e0-3d1c-4e6b-ad85-0956edc88ec3 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:31.253955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:31.273026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d31d0e0-3d1c-4e6b-ad85-0956edc88ec3 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:31.308787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa9ac5-d81c-43f4-90c6-570c6f53d2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:31.309674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa9ac5-d81c-43f4-90c6-570c6f53d2cf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:31.309966 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa9ac5-d81c-43f4-90c6-570c6f53d2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:31.310247 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa9ac5-d81c-43f4-90c6-570c6f53d2cf None admin] PATH_INFO: `/v3/projects/d1956deab85f40baae0279db7a93a3df` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:31.311521 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa9ac5-d81c-43f4-90c6-570c6f53d2cf None admin] RBAC: Authorizing `identity:get_project(project_id=d1956deab85f40baae0279db7a93a3df)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:31.312969 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa9ac5-d81c-43f4-90c6-570c6f53d2cf None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:31.314724 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 594/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 30 18:17:31 2022] GET /identity/v3/projects/d1956deab85f40baae0279db7a93a3df => generated 361 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:31.321176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a0b94-1c36-4fd9-b069-b3e256e05b97 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:31.418998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a0b94-1c36-4fd9-b069-b3e256e05b97 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:31.453817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6625e589-dff5-4975-9bac-d5a51a2ae3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:31.454528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625e589-dff5-4975-9bac-d5a51a2ae3dd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:31.454933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625e589-dff5-4975-9bac-d5a51a2ae3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:31.455269 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625e589-dff5-4975-9bac-d5a51a2ae3dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:31.456124 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6625e589-dff5-4975-9bac-d5a51a2ae3dd None admin] RBAC: Adding query filter params (role.id=0ef3687af20a443281e3cfdceca3597d, scope.project.id=d1956deab85f40baae0279db7a93a3df, user.id=77b615708855457e870683bda0894718) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:17:31.456441 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6625e589-dff5-4975-9bac-d5a51a2ae3dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:31.458129 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6625e589-dff5-4975-9bac-d5a51a2ae3dd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:31.465212 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 606/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:17:31 2022] GET /identity/v3/role_assignments?user.id=77b615708855457e870683bda0894718&scope.project.id=d1956deab85f40baae0279db7a93a3df&role.id=0ef3687af20a443281e3cfdceca3597d => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:17:41.303771 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=25 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:41.461448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:28.565604 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9013542-6b3c-45ab-8cd5-c98fcdacbfcc None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:28.565604 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9013542-6b3c-45ab-8cd5-c98fcdacbfcc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:28.565604 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9013542-6b3c-45ab-8cd5-c98fcdacbfcc None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:28.567281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 595/1201] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:18:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:18:28.577170 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8733983-fef4-4eff-9d38-7192a615af3d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:28.577853 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8733983-fef4-4eff-9d38-7192a615af3d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:28.577853 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8733983-fef4-4eff-9d38-7192a615af3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:28.600406 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f8733983-fef4-4eff-9d38-7192a615af3d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:18:28.601615 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f8733983-fef4-4eff-9d38-7192a615af3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:18:28.699161 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 607/1202] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:18:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:18:28.727492 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9013542-6b3c-45ab-8cd5-c98fcdacbfcc None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:28.853981 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9013542-6b3c-45ab-8cd5-c98fcdacbfcc None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:28.902098 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85befef1-ea8b-41b0-8373-13efc37507bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:28.902998 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85befef1-ea8b-41b0-8373-13efc37507bf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:28.903287 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85befef1-ea8b-41b0-8373-13efc37507bf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:28.903559 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85befef1-ea8b-41b0-8373-13efc37507bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:28.920666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85befef1-ea8b-41b0-8373-13efc37507bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:28.922215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85befef1-ea8b-41b0-8373-13efc37507bf None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:28.942160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-85befef1-ea8b-41b0-8373-13efc37507bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:28.942160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:28.942160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:28.942160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:28.942160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:28.942160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:28.942160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:28.942160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:28.942160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:28.942160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:28.942160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:28.942160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:28.942160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:18:28.942160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:18:28.942160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:18:28.943348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:18:28.943348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:28.943348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:28.943348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:28.943348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:28.943348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:28.943348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:28.943348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:28.943348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:28.943348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:28.943348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:28.943348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:28.943348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:28.943348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:28.943348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:28.943348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:28.943348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:28.944899 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:18:28.944899 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:18:28.944899 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:28.944899 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:18:28.944899 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 596/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:18:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:28.949808 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8733983-fef4-4eff-9d38-7192a615af3d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:29.096496 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8733983-fef4-4eff-9d38-7192a615af3d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:29.139178 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:29.148632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9ade1-65f4-4cf7-a7f0-762906395d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:29.149368 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9ade1-65f4-4cf7-a7f0-762906395d4c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:29.149653 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9ade1-65f4-4cf7-a7f0-762906395d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:29.149937 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9ade1-65f4-4cf7-a7f0-762906395d4c None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:29.151938 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d9ade1-65f4-4cf7-a7f0-762906395d4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:29.151938 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d9ade1-65f4-4cf7-a7f0-762906395d4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:29.153240 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d9ade1-65f4-4cf7-a7f0-762906395d4c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:29.170805 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 608/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:29.178513 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85befef1-ea8b-41b0-8373-13efc37507bf None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:29.242363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:29.314367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85befef1-ea8b-41b0-8373-13efc37507bf None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:29.362252 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843efad-a52f-4fd3-b77a-9650b92f8e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:29.363072 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843efad-a52f-4fd3-b77a-9650b92f8e76 None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:29.363459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843efad-a52f-4fd3-b77a-9650b92f8e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:29.363858 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843efad-a52f-4fd3-b77a-9650b92f8e76 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:29.364687 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5843efad-a52f-4fd3-b77a-9650b92f8e76 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:29.366770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5843efad-a52f-4fd3-b77a-9650b92f8e76 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:29.392144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 597/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 30 18:18:29 2022] POST /identity/v3/users => generated 251 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:18:31.798519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e1010-3ab6-42fa-9179-67231ad0aca3 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:31.799138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e1010-3ab6-42fa-9179-67231ad0aca3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:31.799235 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e1010-3ab6-42fa-9179-67231ad0aca3 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:31.801112 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 609/1206] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:18:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:18:31.806709 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f58e51-76dc-499f-a056-737419d376f1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:31.807162 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f58e51-76dc-499f-a056-737419d376f1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:31.807377 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f58e51-76dc-499f-a056-737419d376f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:31.826342 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-64f58e51-76dc-499f-a056-737419d376f1 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:18:31.830119 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-64f58e51-76dc-499f-a056-737419d376f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:18:31.912186 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 598/1207] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:18:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:18:31.938851 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e1010-3ab6-42fa-9179-67231ad0aca3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:32.094998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e1010-3ab6-42fa-9179-67231ad0aca3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:32.145487 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b876664-08d0-4a2c-933b-d9c14281d4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:32.146860 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b876664-08d0-4a2c-933b-d9c14281d4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:32.147382 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b876664-08d0-4a2c-933b-d9c14281d4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:32.147862 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b876664-08d0-4a2c-933b-d9c14281d4b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:32.163851 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b876664-08d0-4a2c-933b-d9c14281d4b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:18:32.173451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-9b876664-08d0-4a2c-933b-d9c14281d4b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:18:32.173451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:32.173451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:32.173451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:32.173451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:32.173451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:32.173451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:32.173451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:32.173451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:32.173451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:32.173451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:32.173451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:32.173451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:18:32.173451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:18:32.173451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:18:32.173451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:18:32.174654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:18:32.174654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:18:32.174654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:18:32.174654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:18:32.174654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:32.174654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:32.174654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:32.174654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:32.174654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:32.174654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:32.174654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:32.174654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:32.174654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:32.174654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:32.174654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:32.174654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:32.175584 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:32.175584 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:32.175584 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:32.175584 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:32.175584 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:18:32.175584 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:18:32.175584 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:18:32.175584 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:18:32.175584 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:18:32.175584 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:18:32.175584 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:18:32.175584 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:18:32.175584 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 610/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 30 18:18:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:32.182090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f58e51-76dc-499f-a056-737419d376f1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:32.240470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:32.320316 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f58e51-76dc-499f-a056-737419d376f1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:32.361962 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb281c-5d7b-4704-818b-b1d880342376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:32.362571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb281c-5d7b-4704-818b-b1d880342376 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:32.362856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb281c-5d7b-4704-818b-b1d880342376 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:32.363133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb281c-5d7b-4704-818b-b1d880342376 None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:32.363887 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb281c-5d7b-4704-818b-b1d880342376 None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:32.364201 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb281c-5d7b-4704-818b-b1d880342376 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:32.365614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb281c-5d7b-4704-818b-b1d880342376 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:32.374923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 599/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 30 18:18:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:32.380933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b876664-08d0-4a2c-933b-d9c14281d4b5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:32.443983 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:32.509398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b876664-08d0-4a2c-933b-d9c14281d4b5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:32.550560 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c057453-e793-4453-81ba-5a3d02ff9a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:32.551508 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c057453-e793-4453-81ba-5a3d02ff9a82 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:32.551797 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c057453-e793-4453-81ba-5a3d02ff9a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:32.552081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c057453-e793-4453-81ba-5a3d02ff9a82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:32.564532 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c057453-e793-4453-81ba-5a3d02ff9a82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:32.566080 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c057453-e793-4453-81ba-5a3d02ff9a82 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:32.592314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-0c057453-e793-4453-81ba-5a3d02ff9a82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:32.592314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:32.592314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:32.592314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:32.592314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:32.592314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:32.592314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:32.592314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:32.592314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:32.592314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:32.592314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:32.592314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:32.592314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:18:32.592314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:18:32.592314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:18:32.598319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:18:32.598319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:32.598319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:32.598319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:32.598319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:32.598319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:32.598319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:32.598319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:32.598319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:32.598319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:32.598319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:32.598319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:32.598319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:32.598319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:32.598319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:32.598319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:32.598319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:32.599484 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:18:32.599484 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:18:32.599484 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:32.599484 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:18:32.599484 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 611/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:18:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:18:32.599838 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb281c-5d7b-4704-818b-b1d880342376 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:32.717085 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:32.720125 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb281c-5d7b-4704-818b-b1d880342376 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:32.761810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb38dec9-2eb3-4085-a121-df9f99ab9778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:32.762672 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38dec9-2eb3-4085-a121-df9f99ab9778 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:32.762970 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38dec9-2eb3-4085-a121-df9f99ab9778 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:32.763282 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38dec9-2eb3-4085-a121-df9f99ab9778 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:32.764189 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb38dec9-2eb3-4085-a121-df9f99ab9778 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:32.764570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb38dec9-2eb3-4085-a121-df9f99ab9778 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:32.766363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb38dec9-2eb3-4085-a121-df9f99ab9778 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:32.784466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 600/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:32.789535 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c057453-e793-4453-81ba-5a3d02ff9a82 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:32.911155 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c057453-e793-4453-81ba-5a3d02ff9a82 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:32.913149 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:32.951855 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01220f-6a4b-4f38-ae87-eae065a90ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:32.952860 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01220f-6a4b-4f38-ae87-eae065a90ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:32.953219 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01220f-6a4b-4f38-ae87-eae065a90ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:32.953556 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01220f-6a4b-4f38-ae87-eae065a90ee1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:32.962170 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01220f-6a4b-4f38-ae87-eae065a90ee1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:32.963717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01220f-6a4b-4f38-ae87-eae065a90ee1 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:32.984028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-5d01220f-6a4b-4f38-ae87-eae065a90ee1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 18:18:32.984028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:32.984028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:32.984028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:32.984028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:32.984028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:32.984028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:32.984028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:32.984028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:32.984028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:32.984028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:32.984028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:32.984028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:18:32.984028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:18:32.984028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:18:32.984028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:18:32.985272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:32.985272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:32.985272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:18:32.985272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:18:32.985272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:18:32.985272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:18:32.985272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:32.985272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:32.985272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:32.985272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:32.985272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:32.985272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:32.985272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:32.985272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:32.985272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:32.985272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:32.985272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:32.985272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:32.986500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:32.986500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:32.986500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:18:32.986500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:18:32.986500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:18:32.986500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:18:32.986500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:18:32.986500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:18:32.986500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 18:18:32.986500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:18:32.987679 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 612/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:18:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:18:32.994062 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb38dec9-2eb3-4085-a121-df9f99ab9778 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:33.114958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb38dec9-2eb3-4085-a121-df9f99ab9778 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:33.154671 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27f1db-7109-4a01-bb67-2e93f15dd177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:33.155513 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27f1db-7109-4a01-bb67-2e93f15dd177 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:33.155870 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27f1db-7109-4a01-bb67-2e93f15dd177 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:33.156191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27f1db-7109-4a01-bb67-2e93f15dd177 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:33.157072 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27f1db-7109-4a01-bb67-2e93f15dd177 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:33.157425 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27f1db-7109-4a01-bb67-2e93f15dd177 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:33.158984 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27f1db-7109-4a01-bb67-2e93f15dd177 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:33.180110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 601/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Jun 30 18:18:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:33.186722 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01220f-6a4b-4f38-ae87-eae065a90ee1 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:33.299421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01220f-6a4b-4f38-ae87-eae065a90ee1 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:33.343299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf74f05-28cc-495c-9ddb-bbca0c7be6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:33.344057 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf74f05-28cc-495c-9ddb-bbca0c7be6fa None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:33.344372 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf74f05-28cc-495c-9ddb-bbca0c7be6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:33.344658 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf74f05-28cc-495c-9ddb-bbca0c7be6fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:33.383802 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:33.430118 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf74f05-28cc-495c-9ddb-bbca0c7be6fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:33.432070 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf74f05-28cc-495c-9ddb-bbca0c7be6fa None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:33.560566 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 613/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:18:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 18:18:33.566329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27f1db-7109-4a01-bb67-2e93f15dd177 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:33.587902 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:33.698094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27f1db-7109-4a01-bb67-2e93f15dd177 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:33.738713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d8d23-df62-4f55-8586-ef5a15dea00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:33.739532 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d8d23-df62-4f55-8586-ef5a15dea00c None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:33.739812 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d8d23-df62-4f55-8586-ef5a15dea00c None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:33.740088 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d8d23-df62-4f55-8586-ef5a15dea00c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:33.754692 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d8d23-df62-4f55-8586-ef5a15dea00c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:33.756490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d8d23-df62-4f55-8586-ef5a15dea00c None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:33.784408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-157d8d23-df62-4f55-8586-ef5a15dea00c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:33.784408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:33.784408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:33.784408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:33.784408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:33.784408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:33.784408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:33.784408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:33.784408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:33.784408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:33.784408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:33.784408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:33.784408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:18:33.784408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:18:33.784408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:18:33.785526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:18:33.785526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:33.785526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:33.785526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:33.785526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:33.785526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:33.785526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:33.785526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:33.785526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:33.785526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:33.785526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:33.785526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:33.785526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:33.785526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:33.785526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:33.785526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:33.785526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:33.786570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:18:33.786570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:18:33.786570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:33.786570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:18:33.790513 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 602/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:18:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:18:33.792120 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf74f05-28cc-495c-9ddb-bbca0c7be6fa None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:33.862287 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:33.909483 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf74f05-28cc-495c-9ddb-bbca0c7be6fa None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:33.951150 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410183f-5fcf-4198-9086-c3e26fc63b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:33.951910 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410183f-5fcf-4198-9086-c3e26fc63b70 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:33.952266 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410183f-5fcf-4198-9086-c3e26fc63b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:33.952611 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410183f-5fcf-4198-9086-c3e26fc63b70 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:33.953405 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6410183f-5fcf-4198-9086-c3e26fc63b70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:33.953808 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6410183f-5fcf-4198-9086-c3e26fc63b70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:33.955269 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6410183f-5fcf-4198-9086-c3e26fc63b70 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:33.974710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 614/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:33.975423 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d8d23-df62-4f55-8586-ef5a15dea00c None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:34.052502 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:34.093408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d8d23-df62-4f55-8586-ef5a15dea00c None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:34.134469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520db7b3-9d80-4f33-9e51-098a5edb4046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:34.135443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520db7b3-9d80-4f33-9e51-098a5edb4046 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:34.135957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520db7b3-9d80-4f33-9e51-098a5edb4046 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:34.136373 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520db7b3-9d80-4f33-9e51-098a5edb4046 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:34.143971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520db7b3-9d80-4f33-9e51-098a5edb4046 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:34.145546 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520db7b3-9d80-4f33-9e51-098a5edb4046 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:34.164391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-520db7b3-9d80-4f33-9e51-098a5edb4046 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:18:34.164391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:34.164391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:34.164391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:34.164391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:34.164391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:34.164391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:34.164391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:34.164391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:34.164391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:34.164391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:34.164391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:34.164391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:18:34.164391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:18:34.164391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:18:34.166652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:18:34.166652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:34.166652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:34.166652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:34.166652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:34.166652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:34.166652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:34.166652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:34.166652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:34.166652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:34.166652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:34.166652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:34.166652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:34.166652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:34.166652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:34.166652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:34.166652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:34.168560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:18:34.168560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:18:34.168560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:18:34.168560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:18:34.168560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:18:34.168560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:18:34.168560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:18:34.168560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:18:34.173616 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 603/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:18:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:34.174444 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410183f-5fcf-4198-9086-c3e26fc63b70 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:34.293337 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410183f-5fcf-4198-9086-c3e26fc63b70 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:34.320965 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:34.337313 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b32fd6-348b-4486-bac6-0799125aaaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:34.338012 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b32fd6-348b-4486-bac6-0799125aaaea None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:34.338294 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b32fd6-348b-4486-bac6-0799125aaaea None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:34.338569 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b32fd6-348b-4486-bac6-0799125aaaea None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:34.339938 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b32fd6-348b-4486-bac6-0799125aaaea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:34.340201 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b32fd6-348b-4486-bac6-0799125aaaea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:34.341649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b32fd6-348b-4486-bac6-0799125aaaea None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:34.355066 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 615/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:18:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:34.361247 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520db7b3-9d80-4f33-9e51-098a5edb4046 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:34.491972 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520db7b3-9d80-4f33-9e51-098a5edb4046 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:34.534679 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:34.542189 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a050f1-8c31-4310-876a-7223109242cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:34.543056 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a050f1-8c31-4310-876a-7223109242cf None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:34.543414 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a050f1-8c31-4310-876a-7223109242cf None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:34.543768 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a050f1-8c31-4310-876a-7223109242cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:34.544821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a050f1-8c31-4310-876a-7223109242cf None admin] RBAC: Adding query filter params (role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d, scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b, user.id=49325c1733174e0c98b3475a10c9b5c9) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:34.545225 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a050f1-8c31-4310-876a-7223109242cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:34.546952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a050f1-8c31-4310-876a-7223109242cf None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:34.554164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 604/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:18:34 2022] GET /identity/v3/role_assignments?user.id=49325c1733174e0c98b3475a10c9b5c9&scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b&role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:18:36.656901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ef86f-968d-4b65-8ba6-5d0abda3e8f0 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:36.657455 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ef86f-968d-4b65-8ba6-5d0abda3e8f0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:36.657600 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ef86f-968d-4b65-8ba6-5d0abda3e8f0 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:36.659459 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 616/1220] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:18:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:18:36.665640 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97109f7d-e06d-4614-8b7f-b881719f91b5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:36.666044 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97109f7d-e06d-4614-8b7f-b881719f91b5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:36.666228 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97109f7d-e06d-4614-8b7f-b881719f91b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:36.688352 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-97109f7d-e06d-4614-8b7f-b881719f91b5 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:18:36.689594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-97109f7d-e06d-4614-8b7f-b881719f91b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:18:36.761207 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 605/1221] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:18:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:18:36.784571 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ef86f-968d-4b65-8ba6-5d0abda3e8f0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:36.896037 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ef86f-968d-4b65-8ba6-5d0abda3e8f0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:36.934406 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa5863-fc23-46bb-b9fc-403933325b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:36.935175 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa5863-fc23-46bb-b9fc-403933325b17 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:36.935536 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa5863-fc23-46bb-b9fc-403933325b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:36.935837 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa5863-fc23-46bb-b9fc-403933325b17 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:36.946888 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa5863-fc23-46bb-b9fc-403933325b17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:18:36.954848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-fdfa5863-fc23-46bb-b9fc-403933325b17 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:18:36.954848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:36.954848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:36.954848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:36.954848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:36.954848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:36.954848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:36.954848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:36.954848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:36.954848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:36.954848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:36.954848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:36.954848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:18:36.954848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:18:36.954848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:18:36.954848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:18:36.955883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:18:36.955883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:18:36.955883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:18:36.955883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:18:36.955883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:36.955883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:36.955883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:36.955883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:36.955883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:36.955883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:36.955883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:36.955883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:36.955883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:36.955883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:36.955883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:36.955883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:36.956757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:36.956757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:36.956757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:36.956757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:36.956757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:18:36.956757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:18:36.956757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:18:36.956757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:18:36.956757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:18:36.956757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:18:36.956757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:18:36.956757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:18:36.956757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 617/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 30 18:18:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:36.962479 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97109f7d-e06d-4614-8b7f-b881719f91b5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:37.072524 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97109f7d-e06d-4614-8b7f-b881719f91b5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:37.099169 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:37.111368 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ea8b5-0e7f-4afe-b8ba-5e4bb60e884b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:37.112047 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ea8b5-0e7f-4afe-b8ba-5e4bb60e884b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:37.112334 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ea8b5-0e7f-4afe-b8ba-5e4bb60e884b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:37.112614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ea8b5-0e7f-4afe-b8ba-5e4bb60e884b None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:37.113409 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ea8b5-0e7f-4afe-b8ba-5e4bb60e884b None admin] RBAC: Adding query filter params (name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:37.113742 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ea8b5-0e7f-4afe-b8ba-5e4bb60e884b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:37.115087 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ea8b5-0e7f-4afe-b8ba-5e4bb60e884b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:37.123720 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 606/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 30 18:18:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:37.129632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa5863-fc23-46bb-b9fc-403933325b17 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:37.178655 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:37.241946 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa5863-fc23-46bb-b9fc-403933325b17 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:37.279912 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8029f0-880d-4d80-ab04-1dc36322ba08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:37.280761 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8029f0-880d-4d80-ab04-1dc36322ba08 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:37.281109 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8029f0-880d-4d80-ab04-1dc36322ba08 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:37.281348 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8029f0-880d-4d80-ab04-1dc36322ba08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:37.294415 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8029f0-880d-4d80-ab04-1dc36322ba08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:37.296122 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8029f0-880d-4d80-ab04-1dc36322ba08 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:37.317928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-1b8029f0-880d-4d80-ab04-1dc36322ba08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:37.317928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:37.317928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:37.317928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:37.317928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:37.317928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:37.317928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:37.317928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:37.317928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:37.317928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:37.317928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:37.317928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:37.317928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:18:37.317928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:18:37.317928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:18:37.319801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:18:37.319801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:37.319801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:37.319801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:37.319801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:37.319801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:37.319801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:37.319801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:37.319801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:37.319801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:37.319801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:37.319801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:37.319801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:37.319801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:37.319801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:37.319801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:37.319801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:37.326954 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:18:37.326954 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:18:37.326954 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:37.326954 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:18:37.326954 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 618/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:18:37.327406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ea8b5-0e7f-4afe-b8ba-5e4bb60e884b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:37.434881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ea8b5-0e7f-4afe-b8ba-5e4bb60e884b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:37.475541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a50c12-ccf7-4d5e-b83d-8ce9a4c521c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:37.476203 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a50c12-ccf7-4d5e-b83d-8ce9a4c521c9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:37.476490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a50c12-ccf7-4d5e-b83d-8ce9a4c521c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:37.476770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a50c12-ccf7-4d5e-b83d-8ce9a4c521c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:37.477498 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a50c12-ccf7-4d5e-b83d-8ce9a4c521c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:37.477877 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a50c12-ccf7-4d5e-b83d-8ce9a4c521c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:37.479428 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a50c12-ccf7-4d5e-b83d-8ce9a4c521c9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:37.494999 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 607/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:37.502698 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8029f0-880d-4d80-ab04-1dc36322ba08 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:37.609070 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:37.616866 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8029f0-880d-4d80-ab04-1dc36322ba08 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:37.652899 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c0eb8-9e7b-4f78-99f6-e4d4290549bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:37.653839 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c0eb8-9e7b-4f78-99f6-e4d4290549bb None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:37.654197 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c0eb8-9e7b-4f78-99f6-e4d4290549bb None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:37.654582 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c0eb8-9e7b-4f78-99f6-e4d4290549bb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:37.663164 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c0eb8-9e7b-4f78-99f6-e4d4290549bb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:37.664743 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c0eb8-9e7b-4f78-99f6-e4d4290549bb None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:37.681498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-746c0eb8-9e7b-4f78-99f6-e4d4290549bb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 18:18:37.681498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:37.681498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:37.681498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:37.681498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:37.681498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:37.681498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:37.681498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:37.681498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:37.681498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:37.681498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:37.681498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:37.681498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:18:37.681498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:18:37.681498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:18:37.681498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:18:37.682739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:37.682739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:37.682739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:18:37.682739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:18:37.682739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:18:37.682739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:18:37.682739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:37.682739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:37.682739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:37.682739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:37.682739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:37.682739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:37.682739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:37.682739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:37.682739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:37.682739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:37.682739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:37.682739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:37.683906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:37.683906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:37.683906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:18:37.683906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:18:37.683906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:18:37.683906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:18:37.683906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:18:37.683906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:18:37.683906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 18:18:37.683906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:18:37.683906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 619/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:18:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:37.689143 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a50c12-ccf7-4d5e-b83d-8ce9a4c521c9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:37.745082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:37.808789 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a50c12-ccf7-4d5e-b83d-8ce9a4c521c9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:37.847418 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8c699-8bfd-4725-8b30-dfd40fe55ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:37.848114 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8c699-8bfd-4725-8b30-dfd40fe55ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:37.848406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8c699-8bfd-4725-8b30-dfd40fe55ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:37.848690 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8c699-8bfd-4725-8b30-dfd40fe55ea9 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:37.849534 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8c699-8bfd-4725-8b30-dfd40fe55ea9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:37.849888 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8c699-8bfd-4725-8b30-dfd40fe55ea9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:37.851332 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8c699-8bfd-4725-8b30-dfd40fe55ea9 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:37.869837 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 608/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Jun 30 18:18:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:37.876172 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c0eb8-9e7b-4f78-99f6-e4d4290549bb None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:37.986231 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c0eb8-9e7b-4f78-99f6-e4d4290549bb None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:38.032155 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc26c65-4358-4018-b869-be421b7728aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:38.032969 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc26c65-4358-4018-b869-be421b7728aa None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:38.033266 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc26c65-4358-4018-b869-be421b7728aa None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:38.033562 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc26c65-4358-4018-b869-be421b7728aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:38.045725 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc26c65-4358-4018-b869-be421b7728aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:38.047201 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc26c65-4358-4018-b869-be421b7728aa None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:38.065776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-cbc26c65-4358-4018-b869-be421b7728aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:38.065776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:38.065776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:38.065776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:38.065776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:38.065776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:38.065776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:38.065776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:38.065776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:38.065776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:38.065776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:38.065776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:38.065776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:18:38.065776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:18:38.065776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:18:38.066916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:18:38.066916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:38.066916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:38.066916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:38.066916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:38.066916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:38.066916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:38.066916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:38.066916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:38.066916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:38.066916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:38.066916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:38.066916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:38.066916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:38.066916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:38.066916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:38.066916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:38.068192 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:18:38.068192 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:18:38.068192 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:38.068192 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:18:38.068192 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 620/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:38.073261 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8c699-8bfd-4725-8b30-dfd40fe55ea9 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:38.195600 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8c699-8bfd-4725-8b30-dfd40fe55ea9 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:38.202404 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:38.241218 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f2bf8-e24b-4cb3-90f9-4a2faa0f663b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:38.241920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f2bf8-e24b-4cb3-90f9-4a2faa0f663b None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:38.242213 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f2bf8-e24b-4cb3-90f9-4a2faa0f663b None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:38.242500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f2bf8-e24b-4cb3-90f9-4a2faa0f663b None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:38.243242 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f2bf8-e24b-4cb3-90f9-4a2faa0f663b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:38.243542 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f2bf8-e24b-4cb3-90f9-4a2faa0f663b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:38.245023 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f2bf8-e24b-4cb3-90f9-4a2faa0f663b None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:38.258706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 609/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:18:38.264978 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc26c65-4358-4018-b869-be421b7728aa None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:38.275222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:38.381275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc26c65-4358-4018-b869-be421b7728aa None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:38.420805 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d08e5-169a-46fb-b35c-f4f7dbac735b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:38.421712 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d08e5-169a-46fb-b35c-f4f7dbac735b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:38.422109 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d08e5-169a-46fb-b35c-f4f7dbac735b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:38.422472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d08e5-169a-46fb-b35c-f4f7dbac735b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:38.430930 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d08e5-169a-46fb-b35c-f4f7dbac735b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:38.432844 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d08e5-169a-46fb-b35c-f4f7dbac735b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:38.453672 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-ee8d08e5-169a-46fb-b35c-f4f7dbac735b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:18:38.453672 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:38.453672 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:38.453672 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:38.453672 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:38.453672 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:38.453672 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:38.453672 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:38.453672 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:38.453672 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:38.453672 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:38.453672 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:38.453672 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:18:38.453672 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:18:38.453672 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:18:38.454917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:18:38.454917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:38.454917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:38.454917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:38.454917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:38.454917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:38.454917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:38.454917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:38.454917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:38.454917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:38.454917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:38.454917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:38.454917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:38.454917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:38.454917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:38.454917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:38.454917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:38.456017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:18:38.456017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:18:38.456017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:18:38.456017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:18:38.456017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:18:38.456017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:18:38.456017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:18:38.456017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:18:38.456017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 621/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:18:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:38.461387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f2bf8-e24b-4cb3-90f9-4a2faa0f663b None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:38.575988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f2bf8-e24b-4cb3-90f9-4a2faa0f663b None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:38.614046 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb0422-1d7c-4a98-a34f-9bf45a9162e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:38.614708 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb0422-1d7c-4a98-a34f-9bf45a9162e5 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:38.614988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb0422-1d7c-4a98-a34f-9bf45a9162e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:38.615265 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb0422-1d7c-4a98-a34f-9bf45a9162e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:38.616048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb0422-1d7c-4a98-a34f-9bf45a9162e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:38.616356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb0422-1d7c-4a98-a34f-9bf45a9162e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:38.617824 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb0422-1d7c-4a98-a34f-9bf45a9162e5 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:38.637187 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 610/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:18:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:38.643828 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d08e5-169a-46fb-b35c-f4f7dbac735b None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:38.722768 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:38.758414 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d08e5-169a-46fb-b35c-f4f7dbac735b None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:38.796879 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03db15-e42c-4e3c-bb72-d8ecb4e53839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:38.797838 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03db15-e42c-4e3c-bb72-d8ecb4e53839 None admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:38.798244 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03db15-e42c-4e3c-bb72-d8ecb4e53839 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:38.798497 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03db15-e42c-4e3c-bb72-d8ecb4e53839 None admin] PATH_INFO: `/v3/projects/8c7e1ee12aa14b7ab13e07822826b25b/users/49325c1733174e0c98b3475a10c9b5c9/roles/1ccfbd20ff5f4d068de0fe3bf93ebe6d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:38.816746 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03db15-e42c-4e3c-bb72-d8ecb4e53839 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c7e1ee12aa14b7ab13e07822826b25b, user_id=49325c1733174e0c98b3475a10c9b5c9, role_id=1ccfbd20ff5f4d068de0fe3bf93ebe6d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:38.818335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03db15-e42c-4e3c-bb72-d8ecb4e53839 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:38.834136 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 622/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:18:38 2022] PUT /identity/v3/projects/8c7e1ee12aa14b7ab13e07822826b25b/users/49325c1733174e0c98b3475a10c9b5c9/roles/1ccfbd20ff5f4d068de0fe3bf93ebe6d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:18:40.943391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16bb429-b813-4227-931f-a26a659be127 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:40.943967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16bb429-b813-4227-931f-a26a659be127 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:40.943967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16bb429-b813-4227-931f-a26a659be127 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:40.945296 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 611/1233] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:18:40.950621 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e3290-87de-4ea3-835a-7c37cd72fbdb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:40.951104 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e3290-87de-4ea3-835a-7c37cd72fbdb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:40.951198 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e3290-87de-4ea3-835a-7c37cd72fbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:40.968952 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-636e3290-87de-4ea3-835a-7c37cd72fbdb None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:18:40.970616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-636e3290-87de-4ea3-835a-7c37cd72fbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:18:41.047503 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 623/1234] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:18:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:18:41.071610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16bb429-b813-4227-931f-a26a659be127 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:41.098779 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:41.181536 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16bb429-b813-4227-931f-a26a659be127 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:41.220622 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db5a75-82b6-4167-a411-0af3d115ff82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:41.221416 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db5a75-82b6-4167-a411-0af3d115ff82 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:41.221736 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db5a75-82b6-4167-a411-0af3d115ff82 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:41.222023 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db5a75-82b6-4167-a411-0af3d115ff82 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:41.234311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84db5a75-82b6-4167-a411-0af3d115ff82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:18:41.243993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-84db5a75-82b6-4167-a411-0af3d115ff82 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:18:41.243993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:41.243993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:41.243993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:41.243993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:41.243993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:41.243993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:41.243993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:41.243993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:41.243993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:41.243993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:41.243993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:41.243993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:18:41.243993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:18:41.243993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:18:41.243993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:18:41.245135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:18:41.245135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:18:41.245135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:18:41.245135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:18:41.245135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:41.245135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:41.245135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:41.245135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:41.245135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:41.245135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:41.245135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:41.245135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:41.245135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:41.245135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:41.245135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:41.245135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:41.246429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:41.246429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:41.246429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:41.246429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:41.246429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:18:41.246429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:18:41.246429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:18:41.246429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:18:41.246429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:18:41.246429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:18:41.246429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 18:18:41.246429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:18:41.246429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 612/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 30 18:18:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:41.251714 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e3290-87de-4ea3-835a-7c37cd72fbdb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:41.305987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:41.372787 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e3290-87de-4ea3-835a-7c37cd72fbdb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:41.413000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8c8d7-fc84-4d76-98fe-af5aed88530c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:41.413697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8c8d7-fc84-4d76-98fe-af5aed88530c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:41.413979 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8c8d7-fc84-4d76-98fe-af5aed88530c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:41.414258 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8c8d7-fc84-4d76-98fe-af5aed88530c None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:41.414998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8c8d7-fc84-4d76-98fe-af5aed88530c None admin] RBAC: Adding query filter params (name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:41.415306 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8c8d7-fc84-4d76-98fe-af5aed88530c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:41.416680 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8c8d7-fc84-4d76-98fe-af5aed88530c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:41.426184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 624/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 30 18:18:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:41.432986 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db5a75-82b6-4167-a411-0af3d115ff82 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:41.526305 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:41.544192 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db5a75-82b6-4167-a411-0af3d115ff82 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:41.583600 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b6d55-ba33-44d1-b9ce-8f0d2cac38bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:41.584390 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b6d55-ba33-44d1-b9ce-8f0d2cac38bc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:41.584671 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b6d55-ba33-44d1-b9ce-8f0d2cac38bc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:41.584944 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b6d55-ba33-44d1-b9ce-8f0d2cac38bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:41.595955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b6d55-ba33-44d1-b9ce-8f0d2cac38bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:41.597407 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b6d55-ba33-44d1-b9ce-8f0d2cac38bc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:41.618389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c20b6d55-ba33-44d1-b9ce-8f0d2cac38bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:41.618389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:41.618389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:41.618389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:41.618389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:41.618389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:41.618389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:41.618389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:41.618389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:41.618389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:41.618389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:41.618389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:41.618389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:18:41.618389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:18:41.618389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:18:41.619571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:18:41.619571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:41.619571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:41.619571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:41.619571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:41.619571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:41.619571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:41.619571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:41.619571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:41.619571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:41.619571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:41.619571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:41.619571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:41.619571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:41.619571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:41.619571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:41.619571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:41.621906 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:18:41.621906 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:18:41.621906 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:41.621906 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:18:41.621906 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 613/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:18:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:18:41.626270 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8c8d7-fc84-4d76-98fe-af5aed88530c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:41.759925 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8c8d7-fc84-4d76-98fe-af5aed88530c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:41.771205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:41.798933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c130c54-437c-4f75-8e91-fcffbdc1e4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:41.799524 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c130c54-437c-4f75-8e91-fcffbdc1e4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:41.799818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c130c54-437c-4f75-8e91-fcffbdc1e4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:41.800103 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c130c54-437c-4f75-8e91-fcffbdc1e4b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:41.800851 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c130c54-437c-4f75-8e91-fcffbdc1e4b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:41.801279 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c130c54-437c-4f75-8e91-fcffbdc1e4b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:41.802762 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c130c54-437c-4f75-8e91-fcffbdc1e4b5 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:41.820092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 625/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:18:41.821404 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b6d55-ba33-44d1-b9ce-8f0d2cac38bc None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:41.927828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b6d55-ba33-44d1-b9ce-8f0d2cac38bc None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:41.964292 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fd664-5233-4944-a4bf-1bc31d71d71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:41.965101 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fd664-5233-4944-a4bf-1bc31d71d71d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:41.965386 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fd664-5233-4944-a4bf-1bc31d71d71d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:41.965698 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fd664-5233-4944-a4bf-1bc31d71d71d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:41.968583 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:41.974763 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17fd664-5233-4944-a4bf-1bc31d71d71d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:41.976254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17fd664-5233-4944-a4bf-1bc31d71d71d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:41.991702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-d17fd664-5233-4944-a4bf-1bc31d71d71d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 18:18:41.991702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:41.991702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:41.991702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:41.991702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:41.991702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:41.991702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:41.991702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:41.991702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:41.991702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:41.991702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:41.991702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:41.991702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:18:41.991702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:18:41.991702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:18:41.991702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:18:41.992840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:41.992840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:41.992840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:18:41.992840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:18:41.992840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:18:41.992840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:18:41.992840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:41.992840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:41.992840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:41.992840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:41.992840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:41.992840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:41.992840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:41.992840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:41.992840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:41.992840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:41.992840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:41.992840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:41.993900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:41.993900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:41.993900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:18:41.993900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:18:41.993900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:18:41.993900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:18:41.993900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:18:41.993900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:18:41.993900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 18:18:41.993900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:18:41.993900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 614/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:18:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:41.999858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c130c54-437c-4f75-8e91-fcffbdc1e4b5 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:42.118931 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c130c54-437c-4f75-8e91-fcffbdc1e4b5 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:42.158704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe04e6-6ab5-49a5-b46e-68906595dfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:42.159447 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe04e6-6ab5-49a5-b46e-68906595dfa7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:42.159747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe04e6-6ab5-49a5-b46e-68906595dfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:42.160051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe04e6-6ab5-49a5-b46e-68906595dfa7 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:42.160998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe04e6-6ab5-49a5-b46e-68906595dfa7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:42.161348 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe04e6-6ab5-49a5-b46e-68906595dfa7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:42.162967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe04e6-6ab5-49a5-b46e-68906595dfa7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:42.184318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 626/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Jun 30 18:18:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:42.190648 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fd664-5233-4944-a4bf-1bc31d71d71d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:42.312027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fd664-5233-4944-a4bf-1bc31d71d71d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:42.350813 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32effbb-72a6-41e3-b024-41553ccdaaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:42.351610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32effbb-72a6-41e3-b024-41553ccdaaf8 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:42.351900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32effbb-72a6-41e3-b024-41553ccdaaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:42.352180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32effbb-72a6-41e3-b024-41553ccdaaf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:42.424437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:42.433751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32effbb-72a6-41e3-b024-41553ccdaaf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:42.435280 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32effbb-72a6-41e3-b024-41553ccdaaf8 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:42.558039 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 615/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:18:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 18:18:42.563923 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe04e6-6ab5-49a5-b46e-68906595dfa7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:42.594834 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:42.681541 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe04e6-6ab5-49a5-b46e-68906595dfa7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:42.722812 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71885bfd-3d9d-41f9-a941-4b709e49748f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:42.724421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71885bfd-3d9d-41f9-a941-4b709e49748f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:42.724719 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71885bfd-3d9d-41f9-a941-4b709e49748f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:42.725058 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71885bfd-3d9d-41f9-a941-4b709e49748f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:42.739719 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71885bfd-3d9d-41f9-a941-4b709e49748f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:42.741231 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71885bfd-3d9d-41f9-a941-4b709e49748f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:42.759511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-71885bfd-3d9d-41f9-a941-4b709e49748f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:42.759511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:42.759511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:42.759511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:42.759511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:42.759511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:42.759511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:42.759511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:42.759511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:42.759511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:42.759511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:42.759511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:42.759511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:18:42.759511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:18:42.759511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:18:42.766131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:18:42.766131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:42.766131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:42.766131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:42.766131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:42.766131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:42.766131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:42.766131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:42.766131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:42.766131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:42.766131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:42.766131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:42.766131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:42.766131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:42.766131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:42.766131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:42.766131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:42.767241 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:18:42.767241 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:18:42.767241 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:42.767241 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:18:42.767241 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 627/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:18:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:42.767592 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32effbb-72a6-41e3-b024-41553ccdaaf8 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:42.879409 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:42.885717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32effbb-72a6-41e3-b024-41553ccdaaf8 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:42.927747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b3502-c1f5-4b5a-9774-b42c4f17c48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:42.928402 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b3502-c1f5-4b5a-9774-b42c4f17c48e None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:42.928679 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b3502-c1f5-4b5a-9774-b42c4f17c48e None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:42.928950 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b3502-c1f5-4b5a-9774-b42c4f17c48e None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:42.929734 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b3502-c1f5-4b5a-9774-b42c4f17c48e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:42.930297 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b3502-c1f5-4b5a-9774-b42c4f17c48e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:42.931801 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b3502-c1f5-4b5a-9774-b42c4f17c48e None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:42.944684 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 616/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:42.951103 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71885bfd-3d9d-41f9-a941-4b709e49748f None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:43.043901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:43.069955 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71885bfd-3d9d-41f9-a941-4b709e49748f None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:43.112722 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666abf42-88f9-4eba-bb49-cf10ce3dd2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:43.113522 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666abf42-88f9-4eba-bb49-cf10ce3dd2fe None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:43.113840 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666abf42-88f9-4eba-bb49-cf10ce3dd2fe None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:43.114122 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666abf42-88f9-4eba-bb49-cf10ce3dd2fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:43.123494 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666abf42-88f9-4eba-bb49-cf10ce3dd2fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:43.124960 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666abf42-88f9-4eba-bb49-cf10ce3dd2fe None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:43.140125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-666abf42-88f9-4eba-bb49-cf10ce3dd2fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:18:43.140125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:43.140125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:43.140125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:43.140125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:43.140125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:43.140125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:43.140125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:43.140125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:43.140125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:43.140125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:43.140125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:43.140125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:18:43.140125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:18:43.140125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:18:43.146491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:18:43.146491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:43.146491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:43.146491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:43.146491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:43.146491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:43.146491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:43.146491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:43.146491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:43.146491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:43.146491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:43.146491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:43.146491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:43.146491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:43.146491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:43.146491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:43.146491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:43.147626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:18:43.147626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:18:43.147626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:18:43.147626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:18:43.147626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:18:43.147626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:18:43.147626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:18:43.147626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:18:43.147626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 628/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:18:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:43.148231 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b3502-c1f5-4b5a-9774-b42c4f17c48e None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:43.263824 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b3502-c1f5-4b5a-9774-b42c4f17c48e None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:43.303896 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45638332-65af-4283-9d22-06b02f0e242d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:43.304583 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45638332-65af-4283-9d22-06b02f0e242d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:43.305186 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45638332-65af-4283-9d22-06b02f0e242d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:43.305391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45638332-65af-4283-9d22-06b02f0e242d None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:43.306216 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45638332-65af-4283-9d22-06b02f0e242d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:43.306523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45638332-65af-4283-9d22-06b02f0e242d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:43.308473 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45638332-65af-4283-9d22-06b02f0e242d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:43.322106 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 617/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:18:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:43.329875 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666abf42-88f9-4eba-bb49-cf10ce3dd2fe None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:43.454354 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666abf42-88f9-4eba-bb49-cf10ce3dd2fe None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:43.495977 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481e863-75f7-4081-b4a6-a8fa7649fbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:43.496845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481e863-75f7-4081-b4a6-a8fa7649fbaf None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:43.497170 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481e863-75f7-4081-b4a6-a8fa7649fbaf None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:43.497514 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481e863-75f7-4081-b4a6-a8fa7649fbaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:43.498472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481e863-75f7-4081-b4a6-a8fa7649fbaf None admin] RBAC: Adding query filter params (role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d, scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b, user.id=49325c1733174e0c98b3475a10c9b5c9) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:43.498937 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481e863-75f7-4081-b4a6-a8fa7649fbaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:43.500755 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481e863-75f7-4081-b4a6-a8fa7649fbaf None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:43.508534 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 629/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:18:43 2022] GET /identity/v3/role_assignments?user.id=49325c1733174e0c98b3475a10c9b5c9&scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b&role.id=1ccfbd20ff5f4d068de0fe3bf93ebe6d => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:45.809621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df25aa7-034a-4ed9-a032-50581a7c02c6 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:45.809621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df25aa7-034a-4ed9-a032-50581a7c02c6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:45.809621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df25aa7-034a-4ed9-a032-50581a7c02c6 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:45.810705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 618/1247] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:18:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:18:45.819041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f2321b-d4c5-4805-abf9-5bceaebac452 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:45.819575 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f2321b-d4c5-4805-abf9-5bceaebac452 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:45.819575 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f2321b-d4c5-4805-abf9-5bceaebac452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:45.825614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:45.840224 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a9f2321b-d4c5-4805-abf9-5bceaebac452 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:18:45.841770 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a9f2321b-d4c5-4805-abf9-5bceaebac452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:18:45.920272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 630/1248] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:18:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 18:18:45.945219 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df25aa7-034a-4ed9-a032-50581a7c02c6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:45.967815 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:46.073531 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df25aa7-034a-4ed9-a032-50581a7c02c6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:46.111408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27fecef-eab5-47a4-b0f6-d09c262f8ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:46.112231 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fecef-eab5-47a4-b0f6-d09c262f8ead None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:46.112510 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fecef-eab5-47a4-b0f6-d09c262f8ead None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:46.112824 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fecef-eab5-47a4-b0f6-d09c262f8ead None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:46.125899 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27fecef-eab5-47a4-b0f6-d09c262f8ead None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:18:46.135141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-c27fecef-eab5-47a4-b0f6-d09c262f8ead None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:18:46.135141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:46.135141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:46.135141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:46.135141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:46.135141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:46.135141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:46.135141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:46.135141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:46.135141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:46.135141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:46.135141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:46.135141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:18:46.135141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:18:46.135141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:18:46.135141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:18:46.136299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:18:46.136299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:18:46.136299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:18:46.136299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:18:46.136299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:46.136299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:46.136299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:46.136299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:46.136299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:46.136299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:46.136299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:46.136299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:46.136299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:46.136299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:46.136299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:46.136299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:46.137459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:46.137459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:46.137459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:46.137459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:46.137459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:18:46.137459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:18:46.137459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:18:46.137459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:18:46.137459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:18:46.137459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:18:46.137459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:18:46.137459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:18:46.137459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 619/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:18:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 18:18:46.142767 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f2321b-d4c5-4805-abf9-5bceaebac452 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:46.253537 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f2321b-d4c5-4805-abf9-5bceaebac452 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:46.290138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:46.297661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d22103-eefd-42cb-93e4-f554b0616bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:46.298478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d22103-eefd-42cb-93e4-f554b0616bcc None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:46.298895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d22103-eefd-42cb-93e4-f554b0616bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:46.299188 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d22103-eefd-42cb-93e4-f554b0616bcc None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:46.299944 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d22103-eefd-42cb-93e4-f554b0616bcc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:46.300252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d22103-eefd-42cb-93e4-f554b0616bcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:46.301850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d22103-eefd-42cb-93e4-f554b0616bcc None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:46.317039 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 631/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:18:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:46.323311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27fecef-eab5-47a4-b0f6-d09c262f8ead None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:46.425096 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:46.466667 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27fecef-eab5-47a4-b0f6-d09c262f8ead None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:46.511060 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d8b383-90a1-4181-b96b-c18c920c8e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:46.512071 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8b383-90a1-4181-b96b-c18c920c8e04 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:46.512397 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8b383-90a1-4181-b96b-c18c920c8e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:46.512628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8b383-90a1-4181-b96b-c18c920c8e04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:46.524472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d8b383-90a1-4181-b96b-c18c920c8e04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:46.526030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d8b383-90a1-4181-b96b-c18c920c8e04 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:46.543938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-e8d8b383-90a1-4181-b96b-c18c920c8e04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:46.543938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:46.543938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:46.543938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:46.543938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:46.543938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:46.543938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:46.543938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:46.543938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:46.543938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:46.543938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:46.543938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:46.543938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:18:46.543938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:18:46.543938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:18:46.544980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:18:46.544980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:46.544980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:46.544980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:46.544980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:46.544980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:46.544980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:46.544980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:46.544980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:46.544980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:46.544980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:46.544980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:46.544980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:46.544980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:46.544980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:46.544980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:46.544980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:46.545928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:18:46.545928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:18:46.545928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:46.545928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:18:46.545928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 620/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:18:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:46.552380 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d22103-eefd-42cb-93e4-f554b0616bcc None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:46.678835 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d22103-eefd-42cb-93e4-f554b0616bcc None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:46.717802 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac07e92-5cc0-44d2-8b9c-069781a1af33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:46.718472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac07e92-5cc0-44d2-8b9c-069781a1af33 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:46.718753 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac07e92-5cc0-44d2-8b9c-069781a1af33 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:46.719024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac07e92-5cc0-44d2-8b9c-069781a1af33 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:46.719777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac07e92-5cc0-44d2-8b9c-069781a1af33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:46.720121 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac07e92-5cc0-44d2-8b9c-069781a1af33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:46.721517 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac07e92-5cc0-44d2-8b9c-069781a1af33 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:46.735238 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 632/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:46.740992 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d8b383-90a1-4181-b96b-c18c920c8e04 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:46.856548 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d8b383-90a1-4181-b96b-c18c920c8e04 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:46.884570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:46.893762 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f1de3-2472-4eac-a89f-8dcf5811065d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:46.894565 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f1de3-2472-4eac-a89f-8dcf5811065d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:46.894826 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f1de3-2472-4eac-a89f-8dcf5811065d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:46.895097 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f1de3-2472-4eac-a89f-8dcf5811065d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:46.903397 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f1de3-2472-4eac-a89f-8dcf5811065d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:46.904855 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f1de3-2472-4eac-a89f-8dcf5811065d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:46.919131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-dd4f1de3-2472-4eac-a89f-8dcf5811065d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 18:18:46.919131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:46.919131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:46.919131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:46.919131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:46.919131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:46.919131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:46.919131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:46.919131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:46.919131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:46.919131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:46.919131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:46.919131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:18:46.919131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:18:46.919131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:18:46.919131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:18:46.921193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:46.921193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:46.921193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:18:46.921193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:18:46.921193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:18:46.921193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:18:46.921193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:46.921193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:46.921193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:46.921193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:46.921193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:46.921193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:46.921193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:46.921193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:46.921193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:46.921193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:46.921193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:46.921193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:46.922630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:46.922630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:46.922630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:18:46.922630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:18:46.922630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:18:46.922630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:18:46.922630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:18:46.922630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:18:46.922630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 18:18:46.922630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:18:46.922630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 621/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:18:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:46.926544 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac07e92-5cc0-44d2-8b9c-069781a1af33 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:47.004542 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:47.065931 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac07e92-5cc0-44d2-8b9c-069781a1af33 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:47.104065 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76b80c-4960-41d5-8a7b-ad6072c170a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:47.104760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76b80c-4960-41d5-8a7b-ad6072c170a7 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:47.105598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76b80c-4960-41d5-8a7b-ad6072c170a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:47.106049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76b80c-4960-41d5-8a7b-ad6072c170a7 None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:47.106988 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d76b80c-4960-41d5-8a7b-ad6072c170a7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:47.107305 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d76b80c-4960-41d5-8a7b-ad6072c170a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:47.108749 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d76b80c-4960-41d5-8a7b-ad6072c170a7 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:47.126130 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 633/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Jun 30 18:18:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:47.132179 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f1de3-2472-4eac-a89f-8dcf5811065d None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:47.252295 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f1de3-2472-4eac-a89f-8dcf5811065d None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:47.296502 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b03a2b-95b0-4bd8-8a22-384b0dc1e4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:47.297175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b03a2b-95b0-4bd8-8a22-384b0dc1e4c6 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:47.297492 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b03a2b-95b0-4bd8-8a22-384b0dc1e4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:47.297847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b03a2b-95b0-4bd8-8a22-384b0dc1e4c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:47.368890 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:47.384303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b03a2b-95b0-4bd8-8a22-384b0dc1e4c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:47.385812 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b03a2b-95b0-4bd8-8a22-384b0dc1e4c6 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:47.502090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 622/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:18:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 18:18:47.507872 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76b80c-4960-41d5-8a7b-ad6072c170a7 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:47.616132 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76b80c-4960-41d5-8a7b-ad6072c170a7 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:47.634575 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:47.657591 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd6be4-e618-4862-8278-82f3a248680c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:47.658418 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd6be4-e618-4862-8278-82f3a248680c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:47.658706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd6be4-e618-4862-8278-82f3a248680c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:47.659001 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd6be4-e618-4862-8278-82f3a248680c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:47.670208 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd6be4-e618-4862-8278-82f3a248680c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:47.671744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd6be4-e618-4862-8278-82f3a248680c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:47.691009 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-83bd6be4-e618-4862-8278-82f3a248680c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:47.691009 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:47.691009 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:47.691009 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:47.691009 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:47.691009 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:47.691009 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:47.691009 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:47.691009 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:47.691009 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:47.691009 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:47.691009 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:47.691009 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:18:47.691009 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:18:47.691009 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:18:47.692362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:18:47.692362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:47.692362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:47.692362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:47.692362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:47.692362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:47.692362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:47.692362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:47.692362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:47.692362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:47.692362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:47.692362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:47.692362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:47.692362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:47.692362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:47.692362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:47.692362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:47.693747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:18:47.693747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:18:47.693747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:47.693747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:18:47.693747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 634/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:18:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:47.698147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b03a2b-95b0-4bd8-8a22-384b0dc1e4c6 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:47.795443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:47.817324 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b03a2b-95b0-4bd8-8a22-384b0dc1e4c6 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:47.862698 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481b70b-6fbc-4a94-a449-3dffada46667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:47.863385 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481b70b-6fbc-4a94-a449-3dffada46667 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:47.863642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481b70b-6fbc-4a94-a449-3dffada46667 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:47.863925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481b70b-6fbc-4a94-a449-3dffada46667 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:47.864651 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481b70b-6fbc-4a94-a449-3dffada46667 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:47.864963 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481b70b-6fbc-4a94-a449-3dffada46667 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:47.866978 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481b70b-6fbc-4a94-a449-3dffada46667 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:47.884691 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 623/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:47.886006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd6be4-e618-4862-8278-82f3a248680c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:47.997201 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd6be4-e618-4862-8278-82f3a248680c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:48.036307 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a35e0-4dcc-475b-9b9f-6e97a420e36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:48.037144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a35e0-4dcc-475b-9b9f-6e97a420e36c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:48.037437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a35e0-4dcc-475b-9b9f-6e97a420e36c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:48.037850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a35e0-4dcc-475b-9b9f-6e97a420e36c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:48.045988 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a35e0-4dcc-475b-9b9f-6e97a420e36c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:48.047553 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a35e0-4dcc-475b-9b9f-6e97a420e36c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:48.063350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-1f9a35e0-4dcc-475b-9b9f-6e97a420e36c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:18:48.063350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:48.063350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:48.063350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:48.063350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:48.063350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:48.063350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:48.063350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:48.063350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:48.063350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:48.063350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:48.063350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:48.063350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:18:48.063350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:18:48.063350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:18:48.064296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:18:48.064296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:48.064296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:48.064296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:48.064296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:48.064296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:48.064296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:48.064296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:48.064296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:48.064296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:48.064296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:48.064296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:48.064296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:48.064296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:48.064296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:48.064296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:48.064296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:48.065187 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:18:48.065187 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:18:48.065187 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:18:48.065187 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:18:48.065187 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:18:48.065187 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:18:48.065187 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:18:48.065187 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:18:48.069292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 635/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:18:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:48.070399 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481b70b-6fbc-4a94-a449-3dffada46667 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:48.181119 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481b70b-6fbc-4a94-a449-3dffada46667 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:48.222147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e605c65-f451-4b1d-84ee-4fa980850f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:48.222875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e605c65-f451-4b1d-84ee-4fa980850f7d None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:48.223329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e605c65-f451-4b1d-84ee-4fa980850f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:48.223779 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e605c65-f451-4b1d-84ee-4fa980850f7d None admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:48.224785 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e605c65-f451-4b1d-84ee-4fa980850f7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:48.225104 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e605c65-f451-4b1d-84ee-4fa980850f7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:48.227106 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e605c65-f451-4b1d-84ee-4fa980850f7d None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:48.242402 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 624/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:18:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:48.249049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a35e0-4dcc-475b-9b9f-6e97a420e36c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:48.279322 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:48.366619 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a35e0-4dcc-475b-9b9f-6e97a420e36c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:48.408731 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e033c502-ec46-4a77-aa64-4a8a36748a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:48.409629 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033c502-ec46-4a77-aa64-4a8a36748a89 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:48.409963 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033c502-ec46-4a77-aa64-4a8a36748a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:48.410512 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033c502-ec46-4a77-aa64-4a8a36748a89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:48.411549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e033c502-ec46-4a77-aa64-4a8a36748a89 None admin] RBAC: Adding query filter params (role.id=0ef3687af20a443281e3cfdceca3597d, scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b, user.id=49325c1733174e0c98b3475a10c9b5c9) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:48.412005 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e033c502-ec46-4a77-aa64-4a8a36748a89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:48.413794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e033c502-ec46-4a77-aa64-4a8a36748a89 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:48.421972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 636/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:18:48 2022] GET /identity/v3/role_assignments?user.id=49325c1733174e0c98b3475a10c9b5c9&scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b&role.id=0ef3687af20a443281e3cfdceca3597d => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:50.478682 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4e980-dcd0-42e0-ac1c-7bbf7e12091a None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:50.479300 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4e980-dcd0-42e0-ac1c-7bbf7e12091a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:50.479300 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4e980-dcd0-42e0-ac1c-7bbf7e12091a None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:50.481168 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 625/1261] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:18:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:18:50.486610 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efab034-9bd7-473a-ac55-12b117a40715 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:50.487000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efab034-9bd7-473a-ac55-12b117a40715 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:50.487183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efab034-9bd7-473a-ac55-12b117a40715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:50.504317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8efab034-9bd7-473a-ac55-12b117a40715 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:18:50.505531 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8efab034-9bd7-473a-ac55-12b117a40715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:18:50.571898 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 637/1262] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:18:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:18:50.595379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4e980-dcd0-42e0-ac1c-7bbf7e12091a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:50.606705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:50.705697 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4e980-dcd0-42e0-ac1c-7bbf7e12091a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:50.740618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f18d5-451d-4ddd-8622-72c16fbec367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:50.741322 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f18d5-451d-4ddd-8622-72c16fbec367 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:50.741634 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f18d5-451d-4ddd-8622-72c16fbec367 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:50.741917 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f18d5-451d-4ddd-8622-72c16fbec367 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:50.752146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f18d5-451d-4ddd-8622-72c16fbec367 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:18:50.759678 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-0e9f18d5-451d-4ddd-8622-72c16fbec367 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:18:50.759678 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:50.759678 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:50.759678 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:50.759678 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:50.759678 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:50.759678 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:50.759678 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:50.759678 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:50.759678 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:50.759678 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:50.759678 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:50.759678 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:18:50.759678 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:18:50.759678 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:18:50.759678 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:18:50.760810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:18:50.760810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:18:50.760810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:18:50.760810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:18:50.760810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:50.760810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:50.760810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:50.760810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:50.760810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:50.760810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:50.760810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:50.760810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:50.760810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:50.760810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:50.760810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:50.760810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:50.761894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:50.761894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:50.761894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:50.761894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:50.761894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:18:50.761894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:18:50.761894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:18:50.761894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:18:50.761894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:18:50.761894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:18:50.761894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:18:50.761894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:18:50.761894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 626/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:18:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:50.767074 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efab034-9bd7-473a-ac55-12b117a40715 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:50.786029 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:50.864592 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efab034-9bd7-473a-ac55-12b117a40715 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:50.896795 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532eed8b-7850-4f2e-8447-1718d44c6318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:50.897452 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532eed8b-7850-4f2e-8447-1718d44c6318 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:50.897767 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532eed8b-7850-4f2e-8447-1718d44c6318 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:50.898050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532eed8b-7850-4f2e-8447-1718d44c6318 None admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:50.898792 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532eed8b-7850-4f2e-8447-1718d44c6318 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:50.899103 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532eed8b-7850-4f2e-8447-1718d44c6318 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:50.900473 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532eed8b-7850-4f2e-8447-1718d44c6318 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:50.909002 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 638/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:18:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:18:50.914709 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f18d5-451d-4ddd-8622-72c16fbec367 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:50.981270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:51.014426 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f18d5-451d-4ddd-8622-72c16fbec367 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:51.053338 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a382d-595e-4149-8b01-82eb891bac52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:51.054167 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a382d-595e-4149-8b01-82eb891bac52 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:51.054460 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a382d-595e-4149-8b01-82eb891bac52 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:51.054742 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a382d-595e-4149-8b01-82eb891bac52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:51.067754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a382d-595e-4149-8b01-82eb891bac52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:51.069410 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a382d-595e-4149-8b01-82eb891bac52 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:51.087486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-694a382d-595e-4149-8b01-82eb891bac52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:51.087486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:51.087486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:51.087486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:51.087486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:51.087486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:51.087486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:51.087486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:51.087486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:51.087486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:51.087486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:51.087486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:51.087486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:18:51.087486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:18:51.087486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:18:51.088628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:18:51.088628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:51.088628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:51.088628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:51.088628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:51.088628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:51.088628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:51.088628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:51.088628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:51.088628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:51.088628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:51.088628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:51.088628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:51.088628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:51.088628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:51.088628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:51.088628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:51.090353 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:18:51.090353 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:18:51.090353 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:51.090353 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:18:51.090353 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 627/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:18:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:51.096328 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532eed8b-7850-4f2e-8447-1718d44c6318 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:51.175002 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:51.193971 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532eed8b-7850-4f2e-8447-1718d44c6318 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:51.227576 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10a866-a4b0-4877-bc44-0d5e550ccc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:51.228230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10a866-a4b0-4877-bc44-0d5e550ccc29 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:51.228517 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10a866-a4b0-4877-bc44-0d5e550ccc29 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:51.228797 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10a866-a4b0-4877-bc44-0d5e550ccc29 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:51.229511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10a866-a4b0-4877-bc44-0d5e550ccc29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:51.229851 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10a866-a4b0-4877-bc44-0d5e550ccc29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:51.231267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10a866-a4b0-4877-bc44-0d5e550ccc29 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:51.242601 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 639/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:18:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:51.248862 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a382d-595e-4149-8b01-82eb891bac52 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:51.352209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a382d-595e-4149-8b01-82eb891bac52 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:51.372490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:51.390496 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791aeb3-d724-48b6-8f34-69e5f0335349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:51.391303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791aeb3-d724-48b6-8f34-69e5f0335349 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:51.391596 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791aeb3-d724-48b6-8f34-69e5f0335349 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:51.391872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791aeb3-d724-48b6-8f34-69e5f0335349 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:51.400812 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791aeb3-d724-48b6-8f34-69e5f0335349 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:51.402362 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791aeb3-d724-48b6-8f34-69e5f0335349 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:51.418180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-0791aeb3-d724-48b6-8f34-69e5f0335349 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 18:18:51.418180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:51.418180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:51.418180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:51.418180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:51.418180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:51.418180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:51.418180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:51.418180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:51.418180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:51.418180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:51.418180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:51.418180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:18:51.418180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:18:51.418180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:18:51.418180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:18:51.419508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:51.419508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:51.419508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:18:51.419508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:18:51.419508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:18:51.419508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:18:51.419508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:51.419508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:51.419508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:51.419508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:51.419508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:51.419508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:51.419508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:51.419508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:51.419508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:51.419508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:51.419508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:51.419508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:51.420493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:51.420493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:51.420493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:18:51.420493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:18:51.420493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:18:51.420493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:18:51.420493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:18:51.420493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:18:51.420493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 18:18:51.420493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:18:51.420493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 628/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:18:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:51.425195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10a866-a4b0-4877-bc44-0d5e550ccc29 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:51.539180 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10a866-a4b0-4877-bc44-0d5e550ccc29 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:51.572094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e685e7c-0d22-462d-98e2-3e5bbe3f70de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:51.572749 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e685e7c-0d22-462d-98e2-3e5bbe3f70de None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:51.573031 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e685e7c-0d22-462d-98e2-3e5bbe3f70de None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:51.573308 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e685e7c-0d22-462d-98e2-3e5bbe3f70de None admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:51.574849 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e685e7c-0d22-462d-98e2-3e5bbe3f70de None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:51.575156 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e685e7c-0d22-462d-98e2-3e5bbe3f70de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:51.576588 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e685e7c-0d22-462d-98e2-3e5bbe3f70de None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:51.584927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:51.593509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 640/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Jun 30 18:18:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:51.599128 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791aeb3-d724-48b6-8f34-69e5f0335349 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:51.710632 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791aeb3-d724-48b6-8f34-69e5f0335349 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:51.763958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab90dd-226f-43a2-b92b-28e6c89376da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:51.764734 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab90dd-226f-43a2-b92b-28e6c89376da None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:51.765014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab90dd-226f-43a2-b92b-28e6c89376da None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:51.765288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab90dd-226f-43a2-b92b-28e6c89376da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:51.776862 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab90dd-226f-43a2-b92b-28e6c89376da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:51.778506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab90dd-226f-43a2-b92b-28e6c89376da None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:51.795618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-63ab90dd-226f-43a2-b92b-28e6c89376da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:51.795618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:51.795618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:51.795618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:51.795618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:51.795618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:51.795618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:51.795618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:51.795618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:51.795618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:51.795618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:51.795618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:51.795618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:18:51.795618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:18:51.795618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:18:51.796699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:18:51.796699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:51.796699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:51.796699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:51.796699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:51.796699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:51.796699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:51.796699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:51.796699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:51.796699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:51.796699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:51.796699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:51.796699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:51.796699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:51.796699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:51.796699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:51.796699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:51.802240 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:18:51.802240 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:18:51.802240 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:51.802240 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:18:51.802240 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 629/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:18:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:51.802610 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e685e7c-0d22-462d-98e2-3e5bbe3f70de None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:51.900692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e685e7c-0d22-462d-98e2-3e5bbe3f70de None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:51.936805 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c8f36-d764-4139-aefe-dd429be64f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:51.937448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c8f36-d764-4139-aefe-dd429be64f3d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:51.937767 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c8f36-d764-4139-aefe-dd429be64f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:51.938043 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c8f36-d764-4139-aefe-dd429be64f3d None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:51.938755 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c8f36-d764-4139-aefe-dd429be64f3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:51.939059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c8f36-d764-4139-aefe-dd429be64f3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:51.940419 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c8f36-d764-4139-aefe-dd429be64f3d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:51.951563 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 641/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:18:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:51.957616 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab90dd-226f-43a2-b92b-28e6c89376da None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:51.989976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:52.074860 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab90dd-226f-43a2-b92b-28e6c89376da None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:52.109027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82fca1-6885-4323-919a-d2acf417b715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:52.109836 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82fca1-6885-4323-919a-d2acf417b715 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:52.110121 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82fca1-6885-4323-919a-d2acf417b715 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:52.110399 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82fca1-6885-4323-919a-d2acf417b715 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:52.116824 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82fca1-6885-4323-919a-d2acf417b715 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:52.118313 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82fca1-6885-4323-919a-d2acf417b715 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:52.132904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-2b82fca1-6885-4323-919a-d2acf417b715 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:18:52.132904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:52.132904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:52.132904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:52.132904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:52.132904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:52.132904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:52.132904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:52.132904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:52.132904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:52.132904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:52.132904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:52.132904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:18:52.132904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:18:52.132904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:18:52.134036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:18:52.134036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:52.134036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:52.134036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:52.134036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:52.134036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:52.134036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:52.134036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:52.134036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:52.134036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:52.134036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:52.134036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:52.134036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:52.134036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:52.134036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:52.134036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:52.134036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:52.134972 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:18:52.134972 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:18:52.134972 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:18:52.134972 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:18:52.134972 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:18:52.134972 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:18:52.134972 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:18:52.134972 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:18:52.134972 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 630/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:18:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:52.140392 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c8f36-d764-4139-aefe-dd429be64f3d None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:52.199186 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:52.241727 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c8f36-d764-4139-aefe-dd429be64f3d None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:52.279709 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d81728-8ed0-4851-a698-030f79e9059d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:52.280371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d81728-8ed0-4851-a698-030f79e9059d None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:52.280653 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d81728-8ed0-4851-a698-030f79e9059d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:52.280933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d81728-8ed0-4851-a698-030f79e9059d None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:52.281743 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d81728-8ed0-4851-a698-030f79e9059d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:52.282060 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d81728-8ed0-4851-a698-030f79e9059d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:52.283495 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d81728-8ed0-4851-a698-030f79e9059d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:52.296390 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 642/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:18:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:52.302487 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82fca1-6885-4323-919a-d2acf417b715 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:52.395622 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:52.405266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82fca1-6885-4323-919a-d2acf417b715 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:52.452903 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e84c08-839b-4bb4-bd0c-f148d2026a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:52.453834 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e84c08-839b-4bb4-bd0c-f148d2026a49 None admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:52.454120 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e84c08-839b-4bb4-bd0c-f148d2026a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:52.454407 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e84c08-839b-4bb4-bd0c-f148d2026a49 None admin] PATH_INFO: `/v3/projects/8c7e1ee12aa14b7ab13e07822826b25b/users/49325c1733174e0c98b3475a10c9b5c9/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:52.456423 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e84c08-839b-4bb4-bd0c-f148d2026a49 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c7e1ee12aa14b7ab13e07822826b25b, user_id=49325c1733174e0c98b3475a10c9b5c9, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:52.457904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e84c08-839b-4bb4-bd0c-f148d2026a49 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:52.474366 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 631/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 30 18:18:52 2022] PUT /identity/v3/projects/8c7e1ee12aa14b7ab13e07822826b25b/users/49325c1733174e0c98b3475a10c9b5c9/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:18:54.500700 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b9677-59e8-45b8-bd14-0131936e1f9e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:54.501229 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b9677-59e8-45b8-bd14-0131936e1f9e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:54.501357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b9677-59e8-45b8-bd14-0131936e1f9e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:54.502686 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 643/1274] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:18:54.507969 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07572c1-e8ee-40c6-a6cd-b726bb6eb7b3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:54.508373 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07572c1-e8ee-40c6-a6cd-b726bb6eb7b3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:54.508536 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07572c1-e8ee-40c6-a6cd-b726bb6eb7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:54.527269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a07572c1-e8ee-40c6-a6cd-b726bb6eb7b3 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:18:54.528827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a07572c1-e8ee-40c6-a6cd-b726bb6eb7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:18:54.595760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 632/1275] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:18:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:18:54.619543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b9677-59e8-45b8-bd14-0131936e1f9e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:54.722454 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b9677-59e8-45b8-bd14-0131936e1f9e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:54.739156 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:54.760734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51908182-9d05-4563-bc58-04fa83daa7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:54.761452 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51908182-9d05-4563-bc58-04fa83daa7be None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:54.762289 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51908182-9d05-4563-bc58-04fa83daa7be None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:54.762606 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51908182-9d05-4563-bc58-04fa83daa7be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:54.772789 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51908182-9d05-4563-bc58-04fa83daa7be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 18:18:54.781049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-51908182-9d05-4563-bc58-04fa83daa7be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:18:54.781049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:54.781049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:54.781049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:54.781049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:54.781049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:54.781049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:54.781049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:54.781049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:54.781049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:54.781049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:54.781049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:54.781049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 18:18:54.781049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 18:18:54.781049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 18:18:54.781049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 18:18:54.782169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 18:18:54.782169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 18:18:54.782169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 18:18:54.782169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 18:18:54.782169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:54.782169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:54.782169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:54.782169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:54.782169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:54.782169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:54.782169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:54.782169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:54.782169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:54.782169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:54.782169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:54.782169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:54.783079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:54.783079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:54.783079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:54.783079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:54.783079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 18:18:54.783079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 18:18:54.783079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 18:18:54.783079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 18:18:54.783079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 18:18:54.783079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 18:18:54.783079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 18:18:54.783079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:18:54.783079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 644/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 30 18:18:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:54.788267 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07572c1-e8ee-40c6-a6cd-b726bb6eb7b3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:54.865447 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:54.883566 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07572c1-e8ee-40c6-a6cd-b726bb6eb7b3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:54.917067 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1775e-f047-4b86-874b-01d2fb9d67fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:54.917780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1775e-f047-4b86-874b-01d2fb9d67fd None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:54.918131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1775e-f047-4b86-874b-01d2fb9d67fd None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:54.918409 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1775e-f047-4b86-874b-01d2fb9d67fd None admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:54.919144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1775e-f047-4b86-874b-01d2fb9d67fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:54.919443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1775e-f047-4b86-874b-01d2fb9d67fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:54.920805 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1775e-f047-4b86-874b-01d2fb9d67fd None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:54.928628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 633/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 30 18:18:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:54.935126 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51908182-9d05-4563-bc58-04fa83daa7be None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:55.034935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51908182-9d05-4563-bc58-04fa83daa7be None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:55.069897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d202a4a-1716-4ee5-a937-612233e327d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:55.070687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d202a4a-1716-4ee5-a937-612233e327d6 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:55.070972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d202a4a-1716-4ee5-a937-612233e327d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:55.071319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d202a4a-1716-4ee5-a937-612233e327d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:55.082384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d202a4a-1716-4ee5-a937-612233e327d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:55.083859 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d202a4a-1716-4ee5-a937-612233e327d6 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:55.100563 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-9d202a4a-1716-4ee5-a937-612233e327d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:55.100563 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:55.100563 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:55.100563 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:55.100563 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:55.100563 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:55.100563 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:55.100563 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:55.100563 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:55.100563 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:55.100563 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:55.100563 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:55.100563 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:18:55.100563 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:18:55.100563 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:18:55.101738 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:18:55.101738 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:55.101738 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:55.101738 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:55.101738 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:55.101738 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:55.101738 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:55.101738 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:55.101738 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:55.101738 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:55.101738 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:55.101738 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:55.101738 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:55.101738 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:55.101738 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:55.101738 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:55.101738 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:55.102866 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:18:55.102866 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:18:55.102866 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:55.102866 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:18:55.102866 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 645/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:18:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:55.107789 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1775e-f047-4b86-874b-01d2fb9d67fd None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:55.208007 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1775e-f047-4b86-874b-01d2fb9d67fd None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:55.239881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c117e7-1215-408d-8dc4-f154ae1fd315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:55.240527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c117e7-1215-408d-8dc4-f154ae1fd315 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:55.240807 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c117e7-1215-408d-8dc4-f154ae1fd315 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:55.241080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c117e7-1215-408d-8dc4-f154ae1fd315 None admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:55.241814 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c117e7-1215-408d-8dc4-f154ae1fd315 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:55.242122 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c117e7-1215-408d-8dc4-f154ae1fd315 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:55.243481 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c117e7-1215-408d-8dc4-f154ae1fd315 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:55.255273 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 634/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:55.261009 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d202a4a-1716-4ee5-a937-612233e327d6 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:55.292222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:55.362502 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d202a4a-1716-4ee5-a937-612233e327d6 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:55.395792 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a7d07-5c00-4ae7-8761-d5fbd8b2afbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:55.396584 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a7d07-5c00-4ae7-8761-d5fbd8b2afbd None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:55.396869 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a7d07-5c00-4ae7-8761-d5fbd8b2afbd None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:55.397148 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a7d07-5c00-4ae7-8761-d5fbd8b2afbd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:55.404507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a7d07-5c00-4ae7-8761-d5fbd8b2afbd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:55.405979 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a7d07-5c00-4ae7-8761-d5fbd8b2afbd None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:55.420470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-c53a7d07-5c00-4ae7-8761-d5fbd8b2afbd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 18:18:55.420470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:55.420470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:55.420470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:55.420470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:55.420470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:55.420470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:55.420470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:55.420470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:55.420470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:55.420470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:55.420470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:55.420470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 18:18:55.420470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 18:18:55.420470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 18:18:55.420470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 18:18:55.421604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:55.421604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:55.421604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 18:18:55.421604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:18:55.421604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 18:18:55.421604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 18:18:55.421604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:55.421604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:55.421604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:55.421604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:55.421604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:55.421604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:55.421604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:55.421604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:55.421604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:55.421604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:55.421604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:55.421604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:55.422692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:55.422692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:55.422692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 18:18:55.422692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 18:18:55.422692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 18:18:55.422692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 18:18:55.422692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 18:18:55.422692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 18:18:55.422692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 18:18:55.422692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:18:55.422692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 646/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:18:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:55.428141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c117e7-1215-408d-8dc4-f154ae1fd315 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:55.432257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:55.538265 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c117e7-1215-408d-8dc4-f154ae1fd315 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:55.574731 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706302b6-87c6-41e3-b69f-f6c5c77a3229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:55.575306 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706302b6-87c6-41e3-b69f-f6c5c77a3229 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:55.575585 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706302b6-87c6-41e3-b69f-f6c5c77a3229 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:55.575856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706302b6-87c6-41e3-b69f-f6c5c77a3229 None admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:55.576676 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706302b6-87c6-41e3-b69f-f6c5c77a3229 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:55.576980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706302b6-87c6-41e3-b69f-f6c5c77a3229 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:55.578436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706302b6-87c6-41e3-b69f-f6c5c77a3229 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:55.594887 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 635/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Jun 30 18:18:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:55.600833 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a7d07-5c00-4ae7-8761-d5fbd8b2afbd None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:55.691553 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:55.701938 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a7d07-5c00-4ae7-8761-d5fbd8b2afbd None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:55.734674 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e9b83-3a40-41cb-baed-f1e555386f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:55.735314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e9b83-3a40-41cb-baed-f1e555386f95 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:55.735601 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e9b83-3a40-41cb-baed-f1e555386f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:55.735882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e9b83-3a40-41cb-baed-f1e555386f95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:55.808137 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e9b83-3a40-41cb-baed-f1e555386f95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:55.810366 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e9b83-3a40-41cb-baed-f1e555386f95 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:55.895598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:55.905532 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 647/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 30 18:18:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 18:18:55.911422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706302b6-87c6-41e3-b69f-f6c5c77a3229 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:55.970417 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:56.008579 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706302b6-87c6-41e3-b69f-f6c5c77a3229 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:56.041702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d7eeb-8875-4b53-895a-a327960033d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:56.042483 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d7eeb-8875-4b53-895a-a327960033d1 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:56.043446 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d7eeb-8875-4b53-895a-a327960033d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:56.043646 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d7eeb-8875-4b53-895a-a327960033d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:56.053386 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d7eeb-8875-4b53-895a-a327960033d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:56.054853 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d7eeb-8875-4b53-895a-a327960033d1 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:56.071995 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-aa6d7eeb-8875-4b53-895a-a327960033d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:56.071995 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:56.071995 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:56.071995 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:56.071995 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:56.071995 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:56.071995 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:56.071995 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:56.071995 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:56.071995 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:56.071995 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:56.071995 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:56.071995 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 18:18:56.071995 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 18:18:56.071995 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 18:18:56.073080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 18:18:56.073080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:56.073080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:56.073080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:56.073080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:56.073080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:56.073080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:56.073080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:56.073080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:56.073080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:56.073080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:56.073080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:56.073080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:56.073080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:56.073080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:56.073080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:56.073080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:56.074031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 18:18:56.074031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 18:18:56.074031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 18:18:56.074031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:18:56.074031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 636/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:56.079127 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e9b83-3a40-41cb-baed-f1e555386f95 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:56.178960 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e9b83-3a40-41cb-baed-f1e555386f95 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:56.212767 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6838a0-e79d-4aa2-954c-a60247f0f8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:56.213425 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6838a0-e79d-4aa2-954c-a60247f0f8f9 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:56.213746 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6838a0-e79d-4aa2-954c-a60247f0f8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:56.214031 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6838a0-e79d-4aa2-954c-a60247f0f8f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:56.215594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6838a0-e79d-4aa2-954c-a60247f0f8f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:56.215911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6838a0-e79d-4aa2-954c-a60247f0f8f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:56.217722 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6838a0-e79d-4aa2-954c-a60247f0f8f9 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:56.228781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 648/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 30 18:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:56.234464 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d7eeb-8875-4b53-895a-a327960033d1 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:56.336492 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d7eeb-8875-4b53-895a-a327960033d1 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:56.350776 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:56.370153 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a55bb-893c-47ae-af81-729281a24383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:56.370951 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a55bb-893c-47ae-af81-729281a24383 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:56.371236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a55bb-893c-47ae-af81-729281a24383 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:56.371511 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a55bb-893c-47ae-af81-729281a24383 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:56.377902 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a55bb-893c-47ae-af81-729281a24383 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:56.379331 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a55bb-893c-47ae-af81-729281a24383 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:56.393507 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application [None req-444a55bb-893c-47ae-af81-729281a24383 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:18:56.393507 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:56.393507 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:56.393507 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:56.393507 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:56.393507 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:56.393507 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:56.393507 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:56.393507 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:56.393507 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:56.393507 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:56.393507 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:56.393507 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 18:18:56.393507 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 18:18:56.393507 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 18:18:56.394603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 18:18:56.394603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:56.394603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:56.394603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:56.394603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:56.394603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:56.394603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:56.394603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:56.394603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:56.394603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:56.394603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:56.394603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:56.394603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:56.394603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:56.394603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:56.394603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:56.394603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:56.395594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 18:18:56.395594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 18:18:56.395594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 18:18:56.395594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 18:18:56.395594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 18:18:56.395594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 18:18:56.395594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 18:18:56.395594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: ERROR keystone.server.flask.application Jun 30 18:18:56.395594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 637/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 30 18:18:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:56.400469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6838a0-e79d-4aa2-954c-a60247f0f8f9 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:56.454617 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:56.508827 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6838a0-e79d-4aa2-954c-a60247f0f8f9 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:56.542257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f494b76c-1161-4da2-a9a2-6a21a0b7512f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:56.542947 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494b76c-1161-4da2-a9a2-6a21a0b7512f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:56.543236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494b76c-1161-4da2-a9a2-6a21a0b7512f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:56.543521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494b76c-1161-4da2-a9a2-6a21a0b7512f None admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:56.544332 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f494b76c-1161-4da2-a9a2-6a21a0b7512f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:56.544647 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f494b76c-1161-4da2-a9a2-6a21a0b7512f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:56.546128 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f494b76c-1161-4da2-a9a2-6a21a0b7512f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:56.559046 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 649/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 30 18:18:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:56.565161 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a55bb-893c-47ae-af81-729281a24383 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:56.663779 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a55bb-893c-47ae-af81-729281a24383 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:56.696865 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4338c78-7ee5-4eff-94c5-2fbf5cf4c819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:56.697508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4338c78-7ee5-4eff-94c5-2fbf5cf4c819 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:56.697823 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4338c78-7ee5-4eff-94c5-2fbf5cf4c819 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:56.698104 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4338c78-7ee5-4eff-94c5-2fbf5cf4c819 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:56.698929 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4338c78-7ee5-4eff-94c5-2fbf5cf4c819 None admin] RBAC: Adding query filter params (role.id=0ef3687af20a443281e3cfdceca3597d, scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b, user.id=49325c1733174e0c98b3475a10c9b5c9) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:56.699840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4338c78-7ee5-4eff-94c5-2fbf5cf4c819 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:56.701435 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4338c78-7ee5-4eff-94c5-2fbf5cf4c819 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:56.707276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 638/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 30 18:18:56 2022] GET /identity/v3/role_assignments?user.id=49325c1733174e0c98b3475a10c9b5c9&scope.project.id=8c7e1ee12aa14b7ab13e07822826b25b&role.id=0ef3687af20a443281e3cfdceca3597d => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:58.793934 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd139ec-cbda-4a9f-8193-94e51d6b60ea None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:58.794509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd139ec-cbda-4a9f-8193-94e51d6b60ea None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:58.794509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd139ec-cbda-4a9f-8193-94e51d6b60ea None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:58.795874 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 650/1288] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:18:58.801055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae3036-0ed5-4260-b4c6-7a2b9e58d981 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:58.801423 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae3036-0ed5-4260-b4c6-7a2b9e58d981 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:58.801643 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae3036-0ed5-4260-b4c6-7a2b9e58d981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:58.818677 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-17ae3036-0ed5-4260-b4c6-7a2b9e58d981 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:18:58.819853 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-17ae3036-0ed5-4260-b4c6-7a2b9e58d981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:18:58.849052 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:58.885774 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 639/1289] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:18:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:18:58.908836 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd139ec-cbda-4a9f-8193-94e51d6b60ea None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:59.007760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd139ec-cbda-4a9f-8193-94e51d6b60ea None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:59.018854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:59.042579 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf219a-a73c-4db4-97e0-d64747a60000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:59.043676 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf219a-a73c-4db4-97e0-d64747a60000 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:59.044020 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf219a-a73c-4db4-97e0-d64747a60000 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:59.044255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf219a-a73c-4db4-97e0-d64747a60000 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:59.044907 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf219a-a73c-4db4-97e0-d64747a60000 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:59.046347 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf219a-a73c-4db4-97e0-d64747a60000 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:59.058183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-b5bf219a-a73c-4db4-97e0-d64747a60000 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 18:18:59.058183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:59.058183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:18:59.058183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:18:59.058183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:18:59.058183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:18:59.058183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:18:59.058183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:18:59.058183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 18:18:59.058183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:18:59.058183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:18:59.058183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:18:59.058183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:18:59.058183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:18:59.058183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:18:59.058183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:18:59.058183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:18:59.060029 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:18:59.060029 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:18:59.060029 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:18:59.060029 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:18:59.060029 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:18:59.060029 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:18:59.060029 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:18:59.060029 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:18:59.060029 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:18:59.060029 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:18:59.060029 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:18:59.060029 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:18:59.060029 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:18:59.060029 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:18:59.060029 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:18:59.060029 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:18:59.060916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:18:59.060916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:18:59.060916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:18:59.060916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:18:59.060916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:18:59.060916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:18:59.060916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:18:59.060916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:18:59.060916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:18:59.060916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:18:59.060916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:18:59.060916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:18:59.060916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 18:18:59.060916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:18:59.060916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 651/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 30 18:18:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:18:59.066174 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae3036-0ed5-4260-b4c6-7a2b9e58d981 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:59.162373 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae3036-0ed5-4260-b4c6-7a2b9e58d981 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:59.196241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20948fc5-5201-4b12-900c-9ae7088cb563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:59.196883 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20948fc5-5201-4b12-900c-9ae7088cb563 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:59.197162 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20948fc5-5201-4b12-900c-9ae7088cb563 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:59.197435 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20948fc5-5201-4b12-900c-9ae7088cb563 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:59.198159 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20948fc5-5201-4b12-900c-9ae7088cb563 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:59.198463 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20948fc5-5201-4b12-900c-9ae7088cb563 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:59.199853 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20948fc5-5201-4b12-900c-9ae7088cb563 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:59.204615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 640/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Jun 30 18:18:59 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:18:59.210551 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf219a-a73c-4db4-97e0-d64747a60000 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:59.318532 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf219a-a73c-4db4-97e0-d64747a60000 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:59.354254 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26acc8cd-c076-419c-a6fb-a9fbcfbbe04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:59.354911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acc8cd-c076-419c-a6fb-a9fbcfbbe04b None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:59.355193 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acc8cd-c076-419c-a6fb-a9fbcfbbe04b None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:59.356051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acc8cd-c076-419c-a6fb-a9fbcfbbe04b None admin] PATH_INFO: `/v3/services` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:59.357217 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26acc8cd-c076-419c-a6fb-a9fbcfbbe04b None admin] RBAC: Adding query filter params (type=placement) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:59.357524 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26acc8cd-c076-419c-a6fb-a9fbcfbbe04b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:59.358924 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26acc8cd-c076-419c-a6fb-a9fbcfbbe04b None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:59.363321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 652/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Jun 30 18:18:59 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:19:01.419231 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c481254-878b-4664-9a62-3358e7517a68 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:19:01.419783 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c481254-878b-4664-9a62-3358e7517a68 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:19:01.419783 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c481254-878b-4664-9a62-3358e7517a68 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:19:01.421064 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 641/1293] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:19:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:19:01.426111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffd34b-ed57-4fc9-8f1f-d928c7ef3e4e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:19:01.426537 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffd34b-ed57-4fc9-8f1f-d928c7ef3e4e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:19:01.426657 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffd34b-ed57-4fc9-8f1f-d928c7ef3e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:19:01.444889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4cffd34b-ed57-4fc9-8f1f-d928c7ef3e4e None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:19:01.446262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4cffd34b-ed57-4fc9-8f1f-d928c7ef3e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:19:01.479086 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:19:01.521012 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 653/1294] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:19:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:19:01.544365 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c481254-878b-4664-9a62-3358e7517a68 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:19:01.575764 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:19:01.644920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c481254-878b-4664-9a62-3358e7517a68 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:19:01.679277 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18188c03-c26f-4a01-9ad0-cd73cd7a49de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:19:01.679856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18188c03-c26f-4a01-9ad0-cd73cd7a49de None admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:19:01.680139 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18188c03-c26f-4a01-9ad0-cd73cd7a49de None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:19:01.680457 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18188c03-c26f-4a01-9ad0-cd73cd7a49de None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:19:01.681144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18188c03-c26f-4a01-9ad0-cd73cd7a49de None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:19:01.682800 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18188c03-c26f-4a01-9ad0-cd73cd7a49de None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:19:01.695753 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 642/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 30 18:19:01 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:19:03.751023 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3187eb3-6e27-466c-b0ac-e123c02fd26c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:19:03.751579 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3187eb3-6e27-466c-b0ac-e123c02fd26c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:19:03.751579 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3187eb3-6e27-466c-b0ac-e123c02fd26c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:19:03.752968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 654/1296] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:19:03.758190 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46b4fe-3658-4405-ac6a-b16f8fe6f0f9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:19:03.758608 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46b4fe-3658-4405-ac6a-b16f8fe6f0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:19:03.758753 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46b4fe-3658-4405-ac6a-b16f8fe6f0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:19:03.776324 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ae46b4fe-3658-4405-ac6a-b16f8fe6f0f9 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:19:03.778716 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ae46b4fe-3658-4405-ac6a-b16f8fe6f0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:19:03.855456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 643/1297] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:19:03 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:19:03.880583 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3187eb3-6e27-466c-b0ac-e123c02fd26c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:19:03.980462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3187eb3-6e27-466c-b0ac-e123c02fd26c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:19:04.014621 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b8a11-ed81-4f62-b79a-2e6e2aa11c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:19:04.015331 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b8a11-ed81-4f62-b79a-2e6e2aa11c5c None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:19:04.015614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b8a11-ed81-4f62-b79a-2e6e2aa11c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:19:04.015892 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b8a11-ed81-4f62-b79a-2e6e2aa11c5c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:19:04.016535 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403b8a11-ed81-4f62-b79a-2e6e2aa11c5c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:19:04.017935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403b8a11-ed81-4f62-b79a-2e6e2aa11c5c None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:19:04.029704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-403b8a11-ed81-4f62-b79a-2e6e2aa11c5c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 18:19:04.029704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:19:04.029704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:19:04.029704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:19:04.029704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:19:04.029704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:19:04.029704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:19:04.029704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:19:04.029704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 18:19:04.029704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:19:04.029704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:19:04.029704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:19:04.029704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:19:04.029704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:19:04.029704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:19:04.029704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:19:04.029704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:19:04.031006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:19:04.031006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:19:04.031006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:19:04.031006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:19:04.031006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:19:04.031006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:19:04.031006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:19:04.031006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:19:04.031006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:19:04.031006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:19:04.031006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:19:04.031006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:19:04.031006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:19:04.031006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:19:04.031006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:19:04.031006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:19:04.032184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:19:04.032184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:19:04.032184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:19:04.032184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:19:04.032184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:19:04.032184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:19:04.032184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:19:04.032184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:19:04.032184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:19:04.032184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:19:04.032184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:19:04.032184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:19:04.032184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 18:19:04.032184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:19:04.032184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 655/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 30 18:19:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:19:04.037022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae46b4fe-3658-4405-ac6a-b16f8fe6f0f9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:19:04.055546 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:19:04.135505 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae46b4fe-3658-4405-ac6a-b16f8fe6f0f9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:19:04.171820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64328026-98d9-435e-b47a-32ab9f388139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:19:04.172489 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64328026-98d9-435e-b47a-32ab9f388139 None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:19:04.172776 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64328026-98d9-435e-b47a-32ab9f388139 None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:19:04.173053 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64328026-98d9-435e-b47a-32ab9f388139 None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:19:04.173793 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64328026-98d9-435e-b47a-32ab9f388139 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:19:04.174104 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64328026-98d9-435e-b47a-32ab9f388139 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:19:04.175483 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64328026-98d9-435e-b47a-32ab9f388139 None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:19:04.182352 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 644/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Jun 30 18:19:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:19:04.190096 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b8a11-ed81-4f62-b79a-2e6e2aa11c5c None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:19:04.234424 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:19:04.329821 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b8a11-ed81-4f62-b79a-2e6e2aa11c5c None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:19:04.375035 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b9ed7f-278e-4fa2-bb85-cea532c0c95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:19:04.375923 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9ed7f-278e-4fa2-bb85-cea532c0c95f None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:19:04.376324 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9ed7f-278e-4fa2-bb85-cea532c0c95f None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:19:04.376716 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9ed7f-278e-4fa2-bb85-cea532c0c95f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:19:04.377885 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b9ed7f-278e-4fa2-bb85-cea532c0c95f None admin] RBAC: Adding query filter params (interface=public, service_id=b153061cdf054ed58dd7db73593ed125, region_id=RegionOne) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:19:04.378336 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b9ed7f-278e-4fa2-bb85-cea532c0c95f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:19:04.380352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b9ed7f-278e-4fa2-bb85-cea532c0c95f None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:19:04.388030 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 656/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 30 18:19:04 2022] GET /identity/v3/endpoints?service_id=b153061cdf054ed58dd7db73593ed125&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:19:04.393670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64328026-98d9-435e-b47a-32ab9f388139 None admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:19:04.484447 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:19:04.505889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64328026-98d9-435e-b47a-32ab9f388139 None admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:19:04.552351 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc1cd9-fd7a-4407-8da8-e46575bf650f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:19:04.553268 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc1cd9-fd7a-4407-8da8-e46575bf650f None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:19:04.553711 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc1cd9-fd7a-4407-8da8-e46575bf650f None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:19:04.554116 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc1cd9-fd7a-4407-8da8-e46575bf650f None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:19:04.555132 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc1cd9-fd7a-4407-8da8-e46575bf650f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:19:04.557436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc1cd9-fd7a-4407-8da8-e46575bf650f None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:19:04.568440 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 645/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 30 18:19:04 2022] GET /identity/v3/services => generated 2191 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:19:07.151301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b5a0f-d2d3-4f2c-b17f-c23c1ad8f76d None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:19:07.151825 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b5a0f-d2d3-4f2c-b17f-c23c1ad8f76d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:19:07.152119 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b5a0f-d2d3-4f2c-b17f-c23c1ad8f76d None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:19:07.154259 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 657/1302] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:19:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:19:07.160297 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5a4f8-ef55-4b67-b6a3-9dded853b107 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:19:07.160791 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5a4f8-ef55-4b67-b6a3-9dded853b107 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:19:07.160890 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5a4f8-ef55-4b67-b6a3-9dded853b107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:19:07.177962 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-64b5a4f8-ef55-4b67-b6a3-9dded853b107 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:19:07.180209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-64b5a4f8-ef55-4b67-b6a3-9dded853b107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:19:07.247863 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 646/1303] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:19:07 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 18:19:07.271603 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b5a0f-d2d3-4f2c-b17f-c23c1ad8f76d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:19:07.366963 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:19:07.374630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b5a0f-d2d3-4f2c-b17f-c23c1ad8f76d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:19:07.407824 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec6f0a-0232-4a41-ad5d-feab7825a898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:19:07.408611 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec6f0a-0232-4a41-ad5d-feab7825a898 None admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:19:07.408896 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec6f0a-0232-4a41-ad5d-feab7825a898 None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:19:07.409175 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec6f0a-0232-4a41-ad5d-feab7825a898 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:19:07.409861 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec6f0a-0232-4a41-ad5d-feab7825a898 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:19:07.411245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec6f0a-0232-4a41-ad5d-feab7825a898 None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:19:07.423733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application [None req-0eec6f0a-0232-4a41-ad5d-feab7825a898 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 18:19:07.423733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:19:07.423733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 18:19:07.423733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 18:19:07.423733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 18:19:07.423733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 18:19:07.423733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 18:19:07.423733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:19:07.423733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 18:19:07.423733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:19:07.423733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 18:19:07.423733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:19:07.423733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 18:19:07.423733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 18:19:07.423733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 18:19:07.423733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 18:19:07.423733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 18:19:07.424939 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 18:19:07.424939 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 18:19:07.424939 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 18:19:07.424939 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 18:19:07.424939 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 18:19:07.424939 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 18:19:07.424939 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 18:19:07.424939 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 18:19:07.424939 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 18:19:07.424939 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 18:19:07.424939 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 18:19:07.424939 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 18:19:07.424939 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 18:19:07.424939 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 18:19:07.424939 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 18:19:07.424939 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 18:19:07.426205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 18:19:07.426205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application with Lock( Jun 30 18:19:07.426205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 18:19:07.426205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self._enter() Jun 30 18:19:07.426205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 18:19:07.426205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 18:19:07.426205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 18:19:07.426205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application return self.creator() Jun 30 18:19:07.426205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 18:19:07.426205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application created_value = creator( Jun 30 18:19:07.426205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 18:19:07.426205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 18:19:07.426205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 18:19:07.426205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: ERROR keystone.server.flask.application Jun 30 18:19:07.426205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 658/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 30 18:19:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 18:19:07.431243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5a4f8-ef55-4b67-b6a3-9dded853b107 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:19:07.518661 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:19:07.541018 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5a4f8-ef55-4b67-b6a3-9dded853b107 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:19:07.579338 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a692c-1bfb-4e10-a1ba-511e057474dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:19:07.579993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a692c-1bfb-4e10-a1ba-511e057474dc None admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:19:07.580274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a692c-1bfb-4e10-a1ba-511e057474dc None admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:19:07.580544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a692c-1bfb-4e10-a1ba-511e057474dc None admin] PATH_INFO: `/v3/services` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:19:07.581238 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a692c-1bfb-4e10-a1ba-511e057474dc None admin] RBAC: Adding query filter params (name=placement) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:19:07.581541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a692c-1bfb-4e10-a1ba-511e057474dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:19:07.582941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a692c-1bfb-4e10-a1ba-511e057474dc None admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:19:07.587961 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 647/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Jun 30 18:19:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:19:07.593884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec6f0a-0232-4a41-ad5d-feab7825a898 None admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:19:07.696413 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec6f0a-0232-4a41-ad5d-feab7825a898 None admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:19:07.728596 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52a16f-f121-4e5b-b6e4-3ec01d89592d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:19:07.729245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52a16f-f121-4e5b-b6e4-3ec01d89592d None admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:19:07.729535 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52a16f-f121-4e5b-b6e4-3ec01d89592d None admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:19:07.729848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52a16f-f121-4e5b-b6e4-3ec01d89592d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:19:07.730601 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52a16f-f121-4e5b-b6e4-3ec01d89592d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:19:07.732789 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52a16f-f121-4e5b-b6e4-3ec01d89592d None admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:19:07.752271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 659/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 30 18:19:07 2022] POST /identity/v3/endpoints => generated 331 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:19:17.584992 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=30 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:19:17.748241 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:20:12.001316 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af8ecb0-97cb-4e80-b259-37c0b5e34d7d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:12.001974 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af8ecb0-97cb-4e80-b259-37c0b5e34d7d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:12.001974 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af8ecb0-97cb-4e80-b259-37c0b5e34d7d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:12.003458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 648/1307] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:20:12.011411 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61b831-c90c-46c8-9c9b-db5852b4a895 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:12.012396 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61b831-c90c-46c8-9c9b-db5852b4a895 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:12.012890 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61b831-c90c-46c8-9c9b-db5852b4a895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:12.205930 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3b61b831-c90c-46c8-9c9b-db5852b4a895 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:20:12.207440 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3b61b831-c90c-46c8-9c9b-db5852b4a895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:12.479566 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 660/1308] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:20:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:20:12.544421 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f1066-68a5-404a-a5b7-cc56647d0cae None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:12.544778 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f1066-68a5-404a-a5b7-cc56647d0cae None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:12.544989 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f1066-68a5-404a-a5b7-cc56647d0cae None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:12.546674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 649/1309] 10.222.0.33 () {54 vars in 964 bytes} [Thu Jun 30 18:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:20:12.560227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a204a8b-b2eb-4f86-89be-5ed5718256fb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:12.560628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a204a8b-b2eb-4f86-89be-5ed5718256fb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:12.561198 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a204a8b-b2eb-4f86-89be-5ed5718256fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:12.714179 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7a204a8b-b2eb-4f86-89be-5ed5718256fb None None] MFA Rules not processed for user `1d8c54102fb340859ed0acc3189fe376`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:20:12.715526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7a204a8b-b2eb-4f86-89be-5ed5718256fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:12.744498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:20:12.825233 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 661/1310] 10.222.0.33 () {58 vars in 1061 bytes} [Thu Jun 30 18:20:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:20:12.835337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f1066-68a5-404a-a5b7-cc56647d0cae None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:20:12.838175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-763f1066-68a5-404a-a5b7-cc56647d0cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:12.982750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f1066-68a5-404a-a5b7-cc56647d0cae None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:20:13.071325 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905dd2f-7dfd-4646-a25a-3a04376474e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:20:13.071867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905dd2f-7dfd-4646-a25a-3a04376474e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:13.072353 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905dd2f-7dfd-4646-a25a-3a04376474e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:13.072797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905dd2f-7dfd-4646-a25a-3a04376474e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:13.074641 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6905dd2f-7dfd-4646-a25a-3a04376474e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:13.137596 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:20:13.204578 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905dd2f-7dfd-4646-a25a-3a04376474e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:20:13.206397 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905dd2f-7dfd-4646-a25a-3a04376474e1 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:20:13.228188 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 650/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:20:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:20:15.657603 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbc7e7-1540-4078-965c-6984d347c3a5 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:15.657603 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbc7e7-1540-4078-965c-6984d347c3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:15.657603 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbc7e7-1540-4078-965c-6984d347c3a5 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:15.657603 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 662/1312] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:20:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:20:15.658374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026de448-7ec6-4e2f-adcf-c585b95275e0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:15.658374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026de448-7ec6-4e2f-adcf-c585b95275e0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:15.658374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026de448-7ec6-4e2f-adcf-c585b95275e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:15.727532 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-026de448-7ec6-4e2f-adcf-c585b95275e0 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:20:15.728765 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-026de448-7ec6-4e2f-adcf-c585b95275e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:15.836910 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 651/1313] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:20:15.876567 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebbc7e7-1540-4078-965c-6984d347c3a5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:20:15.878292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1ebbc7e7-1540-4078-965c-6984d347c3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:15.988507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebbc7e7-1540-4078-965c-6984d347c3a5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:20:16.061751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:20:16.066834 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38149f-794e-4dd6-837b-fa0e0df0ed3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:20:16.067755 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38149f-794e-4dd6-837b-fa0e0df0ed3d service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:16.068056 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38149f-794e-4dd6-837b-fa0e0df0ed3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:16.068557 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38149f-794e-4dd6-837b-fa0e0df0ed3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:16.070390 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9a38149f-794e-4dd6-837b-fa0e0df0ed3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:16.184548 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38149f-794e-4dd6-837b-fa0e0df0ed3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:20:16.185964 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38149f-794e-4dd6-837b-fa0e0df0ed3d service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:20:16.204218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 663/1314] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:20:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:20:18.682458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee7088-b1e6-4400-bf79-4063432cec16 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:18.683036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee7088-b1e6-4400-bf79-4063432cec16 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:18.683036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee7088-b1e6-4400-bf79-4063432cec16 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:18.684401 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 652/1315] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:20:18.690959 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1b8df-b73a-479f-b123-ab922310d4c2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:18.691354 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1b8df-b73a-479f-b123-ab922310d4c2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:18.691523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1b8df-b73a-479f-b123-ab922310d4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:18.755141 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-bdd1b8df-b73a-479f-b123-ab922310d4c2 None None] MFA Rules not processed for user `e13874941b204c6288bd9759b15662fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:20:18.756373 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bdd1b8df-b73a-479f-b123-ab922310d4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:18.806553 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:20:18.850936 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 664/1316] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:20:18 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:20:18.890628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee7088-b1e6-4400-bf79-4063432cec16 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:20:18.892253 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c2ee7088-b1e6-4400-bf79-4063432cec16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:18.901784 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:20:18.995127 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee7088-b1e6-4400-bf79-4063432cec16 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:20:19.068687 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d47d4b-a039-4177-a922-8134e1f4068a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:20:19.069341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d47d4b-a039-4177-a922-8134e1f4068a service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:19.069666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d47d4b-a039-4177-a922-8134e1f4068a service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:19.069951 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d47d4b-a039-4177-a922-8134e1f4068a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:19.071605 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c6d47d4b-a039-4177-a922-8134e1f4068a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:19.132586 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:20:19.179371 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d47d4b-a039-4177-a922-8134e1f4068a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:20:19.180823 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d47d4b-a039-4177-a922-8134e1f4068a service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:20:19.198907 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 653/1317] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:20:18 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:20:21.814742 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e582d37-1b25-4201-9acf-6b695f58a534 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:21.815315 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e582d37-1b25-4201-9acf-6b695f58a534 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:21.815315 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e582d37-1b25-4201-9acf-6b695f58a534 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:21.816695 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 665/1318] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:20:21.821835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75825555-85e4-421a-a71c-058ec128f1d7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:21.822271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75825555-85e4-421a-a71c-058ec128f1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:21.822442 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75825555-85e4-421a-a71c-058ec128f1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:21.886130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-75825555-85e4-421a-a71c-058ec128f1d7 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:20:21.887372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-75825555-85e4-421a-a71c-058ec128f1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:21.989131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 654/1319] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:20:21 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:20:22.044080 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2fc017-7576-4eed-8048-c9f848744bb5 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:22.044370 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2fc017-7576-4eed-8048-c9f848744bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:22.044683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2fc017-7576-4eed-8048-c9f848744bb5 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:22.046115 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 666/1320] 10.222.0.33 () {54 vars in 964 bytes} [Thu Jun 30 18:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:20:22.052147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dc528-c678-4c16-927d-48a69cc17fc9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:22.052436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dc528-c678-4c16-927d-48a69cc17fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:22.052715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dc528-c678-4c16-927d-48a69cc17fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:22.074154 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:20:22.130531 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b65dc528-c678-4c16-927d-48a69cc17fc9 None None] MFA Rules not processed for user `1d8c54102fb340859ed0acc3189fe376`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:20:22.131759 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b65dc528-c678-4c16-927d-48a69cc17fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:22.209438 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 655/1321] 10.222.0.33 () {58 vars in 1061 bytes} [Thu Jun 30 18:20:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:20:22.218441 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2fc017-7576-4eed-8048-c9f848744bb5 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:20:22.220211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4a2fc017-7576-4eed-8048-c9f848744bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:22.324137 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2fc017-7576-4eed-8048-c9f848744bb5 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:20:22.396687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced13f3-1d8c-42b0-9019-352804d20ee6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:20:22.397241 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced13f3-1d8c-42b0-9019-352804d20ee6 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:22.397526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced13f3-1d8c-42b0-9019-352804d20ee6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:22.397832 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced13f3-1d8c-42b0-9019-352804d20ee6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:22.399457 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8ced13f3-1d8c-42b0-9019-352804d20ee6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:22.406242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:20:22.512159 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced13f3-1d8c-42b0-9019-352804d20ee6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:20:22.513624 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced13f3-1d8c-42b0-9019-352804d20ee6 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:20:22.531279 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 667/1322] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:20:22 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:20:25.745228 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada1b9b-875b-4320-980f-ab1484343750 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:25.745786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada1b9b-875b-4320-980f-ab1484343750 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:25.745906 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada1b9b-875b-4320-980f-ab1484343750 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:25.747292 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 656/1323] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:20:25.752723 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020db74-3cec-43fc-b4a3-2d2c7e8e9bfd None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:25.753132 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020db74-3cec-43fc-b4a3-2d2c7e8e9bfd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:25.753277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020db74-3cec-43fc-b4a3-2d2c7e8e9bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:25.819688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6020db74-3cec-43fc-b4a3-2d2c7e8e9bfd None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:20:25.820884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6020db74-3cec-43fc-b4a3-2d2c7e8e9bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:25.879470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:20:25.923328 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 668/1324] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:20:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:20:25.961870 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada1b9b-875b-4320-980f-ab1484343750 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:20:25.963532 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8ada1b9b-875b-4320-980f-ab1484343750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:26.090514 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada1b9b-875b-4320-980f-ab1484343750 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:20:26.104504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:20:26.167391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e6ccc-802a-4260-a42c-779517d0376e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:20:26.168041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e6ccc-802a-4260-a42c-779517d0376e service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:26.168325 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e6ccc-802a-4260-a42c-779517d0376e service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:26.168619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e6ccc-802a-4260-a42c-779517d0376e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:26.170268 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-828e6ccc-802a-4260-a42c-779517d0376e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:26.279356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e6ccc-802a-4260-a42c-779517d0376e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:20:26.280763 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e6ccc-802a-4260-a42c-779517d0376e service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:20:26.299899 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 657/1325] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:20:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:20:28.411904 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32102b0-abc4-40b4-8dda-f1c9d29bd88a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:28.412426 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32102b0-abc4-40b4-8dda-f1c9d29bd88a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:28.412426 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32102b0-abc4-40b4-8dda-f1c9d29bd88a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:28.413787 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 669/1326] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:20:28.419019 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7794fa0-2d49-4581-bf1e-b9f429a8708c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:28.419444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7794fa0-2d49-4581-bf1e-b9f429a8708c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:28.419590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7794fa0-2d49-4581-bf1e-b9f429a8708c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:28.471989 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:20:28.490581 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e7794fa0-2d49-4581-bf1e-b9f429a8708c None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:20:28.491804 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e7794fa0-2d49-4581-bf1e-b9f429a8708c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:28.590741 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 658/1327] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:20:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:20:28.626931 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32102b0-abc4-40b4-8dda-f1c9d29bd88a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:20:28.628465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e32102b0-abc4-40b4-8dda-f1c9d29bd88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:28.738295 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32102b0-abc4-40b4-8dda-f1c9d29bd88a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:20:28.817460 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c88a30-0ef4-4f84-bddb-3f89d51be19a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:20:28.818169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c88a30-0ef4-4f84-bddb-3f89d51be19a service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:28.818457 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c88a30-0ef4-4f84-bddb-3f89d51be19a service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:28.818745 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c88a30-0ef4-4f84-bddb-3f89d51be19a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:28.820498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-86c88a30-0ef4-4f84-bddb-3f89d51be19a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:28.840852 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:20:28.940767 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c88a30-0ef4-4f84-bddb-3f89d51be19a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:20:28.942253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c88a30-0ef4-4f84-bddb-3f89d51be19a service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:20:28.961125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 670/1328] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:20:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:20:31.084279 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199fbc6-c65d-4e70-943e-0d30b58ba17c None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:31.084905 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199fbc6-c65d-4e70-943e-0d30b58ba17c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:31.084905 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199fbc6-c65d-4e70-943e-0d30b58ba17c None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:31.086261 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 659/1329] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:20:31.091431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accecf76-7213-4569-b3df-03f121203648 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:31.091888 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accecf76-7213-4569-b3df-03f121203648 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:31.092092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accecf76-7213-4569-b3df-03f121203648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:31.163753 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-accecf76-7213-4569-b3df-03f121203648 None None] MFA Rules not processed for user `e13874941b204c6288bd9759b15662fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:20:31.164910 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-accecf76-7213-4569-b3df-03f121203648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:31.232808 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:20:31.272073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 671/1330] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:20:31 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:20:31.309842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199fbc6-c65d-4e70-943e-0d30b58ba17c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:20:31.311365 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a199fbc6-c65d-4e70-943e-0d30b58ba17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:31.420343 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199fbc6-c65d-4e70-943e-0d30b58ba17c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:20:31.442067 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:20:31.498488 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d75ff-6082-4ce9-8b2b-4fc1bcfd43db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:20:31.499162 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d75ff-6082-4ce9-8b2b-4fc1bcfd43db service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:31.499449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d75ff-6082-4ce9-8b2b-4fc1bcfd43db service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:31.499725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d75ff-6082-4ce9-8b2b-4fc1bcfd43db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:31.501916 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8f8d75ff-6082-4ce9-8b2b-4fc1bcfd43db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:31.609448 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d75ff-6082-4ce9-8b2b-4fc1bcfd43db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:20:31.611040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d75ff-6082-4ce9-8b2b-4fc1bcfd43db service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:20:31.645354 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 660/1331] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:20:31 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:20:33.985317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e65f30-adb3-4783-928a-2b7a47477fe4 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:33.985317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e65f30-adb3-4783-928a-2b7a47477fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:33.986379 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e65f30-adb3-4783-928a-2b7a47477fe4 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:33.987146 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 672/1332] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:20:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:20:33.992392 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc54365-e304-4f4e-b339-78afed217d91 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:33.992808 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc54365-e304-4f4e-b339-78afed217d91 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:33.992954 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc54365-e304-4f4e-b339-78afed217d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:34.054934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:20:34.057968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8fc54365-e304-4f4e-b339-78afed217d91 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:20:34.059711 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8fc54365-e304-4f4e-b339-78afed217d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:34.157672 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 661/1333] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:20:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:20:34.193999 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e65f30-adb3-4783-928a-2b7a47477fe4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:20:34.195562 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b7e65f30-adb3-4783-928a-2b7a47477fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:34.308389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e65f30-adb3-4783-928a-2b7a47477fe4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:20:34.383157 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2ce3a-dafa-44f8-bc11-ca5b63e2a625 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:20:34.383796 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2ce3a-dafa-44f8-bc11-ca5b63e2a625 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:34.384078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2ce3a-dafa-44f8-bc11-ca5b63e2a625 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:34.384354 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2ce3a-dafa-44f8-bc11-ca5b63e2a625 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:34.386008 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-60c2ce3a-dafa-44f8-bc11-ca5b63e2a625 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:34.413247 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:20:34.495652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c2ce3a-dafa-44f8-bc11-ca5b63e2a625 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:20:34.497015 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c2ce3a-dafa-44f8-bc11-ca5b63e2a625 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:20:34.514570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 673/1334] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:20:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:20:36.659198 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1a3ec-da08-41cb-a18e-6cc5ae314512 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:36.659761 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1a3ec-da08-41cb-a18e-6cc5ae314512 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:36.659761 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1a3ec-da08-41cb-a18e-6cc5ae314512 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:36.661127 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 662/1335] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:20:36.666471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922f22b-a5f4-4115-b160-9b0cb7b970a7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:36.666915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922f22b-a5f4-4115-b160-9b0cb7b970a7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:36.667040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922f22b-a5f4-4115-b160-9b0cb7b970a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:36.734173 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1922f22b-a5f4-4115-b160-9b0cb7b970a7 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:20:36.736121 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1922f22b-a5f4-4115-b160-9b0cb7b970a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:36.809138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:20:36.832419 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 674/1336] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:20:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:20:36.873272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1a3ec-da08-41cb-a18e-6cc5ae314512 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:20:36.874830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-beb1a3ec-da08-41cb-a18e-6cc5ae314512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:36.982893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1a3ec-da08-41cb-a18e-6cc5ae314512 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:20:37.010181 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:20:37.056642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45af67b-3380-46f0-80a6-c76b48956b4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:20:37.057282 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45af67b-3380-46f0-80a6-c76b48956b4b service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:37.057561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45af67b-3380-46f0-80a6-c76b48956b4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:37.057866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45af67b-3380-46f0-80a6-c76b48956b4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:37.059482 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d45af67b-3380-46f0-80a6-c76b48956b4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:37.166592 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45af67b-3380-46f0-80a6-c76b48956b4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:20:37.168028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45af67b-3380-46f0-80a6-c76b48956b4b service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:20:37.185824 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 663/1337] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:20:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:20:39.619057 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18136a37-496b-4a0d-842b-76faca4bdf2c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:39.619640 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18136a37-496b-4a0d-842b-76faca4bdf2c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:39.619640 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18136a37-496b-4a0d-842b-76faca4bdf2c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:39.621012 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 675/1338] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:20:39.627048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2790e-b0d5-4628-a067-ba1742947b8d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:39.627500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2790e-b0d5-4628-a067-ba1742947b8d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:39.627613 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2790e-b0d5-4628-a067-ba1742947b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:39.702102 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9aa2790e-b0d5-4628-a067-ba1742947b8d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:20:39.703602 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9aa2790e-b0d5-4628-a067-ba1742947b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:39.712968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:20:39.802927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 664/1339] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:20:39 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:20:39.840424 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18136a37-496b-4a0d-842b-76faca4bdf2c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:20:39.841970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-18136a37-496b-4a0d-842b-76faca4bdf2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:39.946654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18136a37-496b-4a0d-842b-76faca4bdf2c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:20:40.026725 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a5ddb1-66b7-4906-b402-81f5e8981d9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:20:40.027253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5ddb1-66b7-4906-b402-81f5e8981d9c service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:40.027538 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5ddb1-66b7-4906-b402-81f5e8981d9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:40.027846 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5ddb1-66b7-4906-b402-81f5e8981d9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:40.029532 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f5a5ddb1-66b7-4906-b402-81f5e8981d9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:40.067788 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:20:40.142044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a5ddb1-66b7-4906-b402-81f5e8981d9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:20:40.143498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a5ddb1-66b7-4906-b402-81f5e8981d9c service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:20:40.161942 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 676/1340] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:20:39 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:20:44.311835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410343d-3c52-444b-a285-fe54dc50a075 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:44.312342 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410343d-3c52-444b-a285-fe54dc50a075 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:44.312342 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410343d-3c52-444b-a285-fe54dc50a075 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:44.315325 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 665/1341] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:20:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:20:44.320766 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346df5bf-cb45-4187-99e4-6d2052238cb9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:44.321243 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346df5bf-cb45-4187-99e4-6d2052238cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:44.321339 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346df5bf-cb45-4187-99e4-6d2052238cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:44.406459 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-346df5bf-cb45-4187-99e4-6d2052238cb9 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:20:44.407796 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-346df5bf-cb45-4187-99e4-6d2052238cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:44.509102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:20:44.524027 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 677/1342] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:20:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:20:44.565147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410343d-3c52-444b-a285-fe54dc50a075 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:20:44.567429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2410343d-3c52-444b-a285-fe54dc50a075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:44.677283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410343d-3c52-444b-a285-fe54dc50a075 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:20:44.681212 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167dbaa0-9339-46d5-acdd-4b93cb9c7687 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:20:44.681894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167dbaa0-9339-46d5-acdd-4b93cb9c7687 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:44.682181 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167dbaa0-9339-46d5-acdd-4b93cb9c7687 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:44.682469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167dbaa0-9339-46d5-acdd-4b93cb9c7687 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:44.693343 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167dbaa0-9339-46d5-acdd-4b93cb9c7687 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:20:44.694828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167dbaa0-9339-46d5-acdd-4b93cb9c7687 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:20:44.705748 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 666/1343] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:20:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:20:47.524837 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf93d0-6b9d-42bc-bf08-8a2faf46432b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:47.525422 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf93d0-6b9d-42bc-bf08-8a2faf46432b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:47.525422 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf93d0-6b9d-42bc-bf08-8a2faf46432b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:47.526854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 678/1344] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:20:47.533297 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6e1ed-dd97-42ef-90d7-8421821f36b4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:47.533909 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6e1ed-dd97-42ef-90d7-8421821f36b4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:47.534027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6e1ed-dd97-42ef-90d7-8421821f36b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:47.565537 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-59e6e1ed-dd97-42ef-90d7-8421821f36b4 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:20:47.566934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-59e6e1ed-dd97-42ef-90d7-8421821f36b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:47.599326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:20:47.655675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 667/1345] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:20:47 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:20:47.696100 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf93d0-6b9d-42bc-bf08-8a2faf46432b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:20:47.708721 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf93d0-6b9d-42bc-bf08-8a2faf46432b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:20:47.712600 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560d820-c972-4594-80cc-0c59991d42db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:20:47.713266 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560d820-c972-4594-80cc-0c59991d42db service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:47.713744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560d820-c972-4594-80cc-0c59991d42db service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:47.714029 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560d820-c972-4594-80cc-0c59991d42db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:47.724804 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b560d820-c972-4594-80cc-0c59991d42db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:20:47.726285 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b560d820-c972-4594-80cc-0c59991d42db service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:20:47.736465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 679/1346] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:20:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:20:52.116233 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff3fa7-d36f-44a0-b949-4a3d49332bb1 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:52.116784 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff3fa7-d36f-44a0-b949-4a3d49332bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:52.116784 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff3fa7-d36f-44a0-b949-4a3d49332bb1 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:52.118259 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 668/1347] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:20:52.123766 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570aa8e-9589-451a-96a3-02f02e1e9a99 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:52.124199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570aa8e-9589-451a-96a3-02f02e1e9a99 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:52.124356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570aa8e-9589-451a-96a3-02f02e1e9a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:52.143404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e570aa8e-9589-451a-96a3-02f02e1e9a99 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:20:52.144657 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e570aa8e-9589-451a-96a3-02f02e1e9a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:52.152857 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 680/1348] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:20:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:20:52.194876 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff3fa7-d36f-44a0-b949-4a3d49332bb1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:20:52.196917 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-13ff3fa7-d36f-44a0-b949-4a3d49332bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:52.210807 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff3fa7-d36f-44a0-b949-4a3d49332bb1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:20:52.214411 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb03e8-2886-467b-ab12-babbc8df55f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:20:52.215050 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb03e8-2886-467b-ab12-babbc8df55f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:52.215339 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb03e8-2886-467b-ab12-babbc8df55f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:52.215623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb03e8-2886-467b-ab12-babbc8df55f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:52.225636 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb03e8-2886-467b-ab12-babbc8df55f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:20:52.227088 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb03e8-2886-467b-ab12-babbc8df55f1 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:20:52.237215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 669/1349] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:20:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:20:57.407142 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755899a-3107-4336-a9ff-b0a319722fce None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:57.408871 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755899a-3107-4336-a9ff-b0a319722fce None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:57.409444 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755899a-3107-4336-a9ff-b0a319722fce None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:57.411439 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 681/1350] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 30 18:20:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:20:57.419728 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99e665-f84f-4045-b2c1-0783d6e84717 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:20:57.420549 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99e665-f84f-4045-b2c1-0783d6e84717 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:20:57.421048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99e665-f84f-4045-b2c1-0783d6e84717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:20:57.534033 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9d99e665-f84f-4045-b2c1-0783d6e84717 None None] MFA Rules not processed for user `49325c1733174e0c98b3475a10c9b5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:20:57.535320 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9d99e665-f84f-4045-b2c1-0783d6e84717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:20:57.654084 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 670/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 30 18:20:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:21:01.912561 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63954b8-6d05-4122-8097-dc535f31b6ce None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:01.913449 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63954b8-6d05-4122-8097-dc535f31b6ce None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:01.913758 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63954b8-6d05-4122-8097-dc535f31b6ce None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:01.915147 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 682/1352] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 30 18:21:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:21:01.920410 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04d488-eafa-4241-9bc8-491d78e0902d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:01.920829 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04d488-eafa-4241-9bc8-491d78e0902d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:01.921042 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04d488-eafa-4241-9bc8-491d78e0902d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:01.941013 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4e04d488-eafa-4241-9bc8-491d78e0902d None None] MFA Rules not processed for user `49325c1733174e0c98b3475a10c9b5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:01.942557 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4e04d488-eafa-4241-9bc8-491d78e0902d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:01.951434 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 671/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 30 18:21:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:21:02.137773 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=13 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:21:03.450616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63153b-1102-4a0b-923f-a06240957873 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:03.451242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63153b-1102-4a0b-923f-a06240957873 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:03.451242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63153b-1102-4a0b-923f-a06240957873 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:03.452891 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 683/1354] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 30 18:21:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:21:03.458040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1144273-6578-43fb-afdd-d4bf44ca3395 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:03.458475 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1144273-6578-43fb-afdd-d4bf44ca3395 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:03.458593 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1144273-6578-43fb-afdd-d4bf44ca3395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:03.477778 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a1144273-6578-43fb-afdd-d4bf44ca3395 None None] MFA Rules not processed for user `49325c1733174e0c98b3475a10c9b5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:03.478974 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a1144273-6578-43fb-afdd-d4bf44ca3395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:03.488345 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 672/1355] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 30 18:21:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:21:07.210013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a2b75-2c3d-46b6-b5cf-779fbe43cc1e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:07.210592 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a2b75-2c3d-46b6-b5cf-779fbe43cc1e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:07.210592 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a2b75-2c3d-46b6-b5cf-779fbe43cc1e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:07.211947 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 684/1356] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:21:07.217285 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01d3ff-5e45-4fd8-85f2-c295824580ca None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:07.217770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01d3ff-5e45-4fd8-85f2-c295824580ca None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:07.217903 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01d3ff-5e45-4fd8-85f2-c295824580ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:07.237044 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fe01d3ff-5e45-4fd8-85f2-c295824580ca None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:07.238454 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fe01d3ff-5e45-4fd8-85f2-c295824580ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:07.249327 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 673/1357] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:21:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:21:07.310419 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff92b64-7deb-414d-8cb8-54b07dbb178e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:07.311025 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff92b64-7deb-414d-8cb8-54b07dbb178e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:07.311557 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff92b64-7deb-414d-8cb8-54b07dbb178e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:07.313475 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 685/1358] 10.222.0.33 () {54 vars in 961 bytes} [Thu Jun 30 18:21:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:21:07.323228 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20f5c6-bf44-45e7-bd2c-68496c640096 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:07.324024 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20f5c6-bf44-45e7-bd2c-68496c640096 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:07.324694 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20f5c6-bf44-45e7-bd2c-68496c640096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:07.372901 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:21:07.382103 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5b20f5c6-bf44-45e7-bd2c-68496c640096 None None] MFA Rules not processed for user `b8751a4e92174d18b7b138392d2579a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:07.383332 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5b20f5c6-bf44-45e7-bd2c-68496c640096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:07.475047 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 674/1359] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Jun 30 18:21:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:21:07.482745 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff92b64-7deb-414d-8cb8-54b07dbb178e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:21:07.497936 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff92b64-7deb-414d-8cb8-54b07dbb178e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:21:07.503980 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805e74f-e520-4c76-95b4-640fcb651328 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:21:07.504620 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805e74f-e520-4c76-95b4-640fcb651328 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:07.504905 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805e74f-e520-4c76-95b4-640fcb651328 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:07.505479 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805e74f-e520-4c76-95b4-640fcb651328 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:07.519214 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805e74f-e520-4c76-95b4-640fcb651328 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:21:07.521559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805e74f-e520-4c76-95b4-640fcb651328 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:21:07.536175 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 686/1360] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:21:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:21:09.819531 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be223b1-6e37-45f7-8513-254cf2200415 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:09.820090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be223b1-6e37-45f7-8513-254cf2200415 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:09.820090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be223b1-6e37-45f7-8513-254cf2200415 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:09.821468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 675/1361] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:21:09.826722 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56809a82-ca06-48da-b1c4-cc0fa8ecb681 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:09.827128 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56809a82-ca06-48da-b1c4-cc0fa8ecb681 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:09.827299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56809a82-ca06-48da-b1c4-cc0fa8ecb681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:09.853457 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-56809a82-ca06-48da-b1c4-cc0fa8ecb681 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:09.854736 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-56809a82-ca06-48da-b1c4-cc0fa8ecb681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:09.864311 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 687/1362] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:21:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:21:09.910613 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be223b1-6e37-45f7-8513-254cf2200415 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:21:09.926647 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be223b1-6e37-45f7-8513-254cf2200415 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:21:09.932642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a97274-78b9-4705-9f51-41eb5d61495a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:21:09.933525 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a97274-78b9-4705-9f51-41eb5d61495a service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:09.934026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a97274-78b9-4705-9f51-41eb5d61495a service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:09.934499 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a97274-78b9-4705-9f51-41eb5d61495a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:09.945442 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a97274-78b9-4705-9f51-41eb5d61495a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:21:09.947227 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a97274-78b9-4705-9f51-41eb5d61495a service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:21:09.958267 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 676/1363] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:21:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:21:12.846321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11b33a-e7fd-47e7-b85d-b9180e2304db None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:12.846843 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11b33a-e7fd-47e7-b85d-b9180e2304db None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:12.846935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11b33a-e7fd-47e7-b85d-b9180e2304db None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:12.848260 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 688/1364] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:21:12.853662 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260516e-0483-45b9-89cb-d04f1c051b20 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:12.854112 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260516e-0483-45b9-89cb-d04f1c051b20 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:12.854211 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260516e-0483-45b9-89cb-d04f1c051b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:12.873032 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0260516e-0483-45b9-89cb-d04f1c051b20 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:12.874284 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0260516e-0483-45b9-89cb-d04f1c051b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:12.893801 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 677/1365] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:21:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:21:12.936059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e496e-53a3-4d31-a15d-99e0dc9daf0b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:12.936335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e496e-53a3-4d31-a15d-99e0dc9daf0b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:12.936617 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e496e-53a3-4d31-a15d-99e0dc9daf0b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:12.938319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 689/1366] 10.222.0.33 () {54 vars in 961 bytes} [Thu Jun 30 18:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:21:12.945637 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cafce8-47da-4cb4-8c36-539b1ebd177e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:12.945851 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cafce8-47da-4cb4-8c36-539b1ebd177e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:12.946065 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cafce8-47da-4cb4-8c36-539b1ebd177e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:12.967649 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c3cafce8-47da-4cb4-8c36-539b1ebd177e None None] MFA Rules not processed for user `b8751a4e92174d18b7b138392d2579a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:12.968781 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c3cafce8-47da-4cb4-8c36-539b1ebd177e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:12.978420 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 678/1367] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Jun 30 18:21:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:21:12.986345 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e496e-53a3-4d31-a15d-99e0dc9daf0b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:21:13.001856 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e496e-53a3-4d31-a15d-99e0dc9daf0b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:21:13.007747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec96e74-121c-40de-8fe4-d1f526915bba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:21:13.008379 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec96e74-121c-40de-8fe4-d1f526915bba service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:13.008692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec96e74-121c-40de-8fe4-d1f526915bba service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:13.008996 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec96e74-121c-40de-8fe4-d1f526915bba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:13.017763 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec96e74-121c-40de-8fe4-d1f526915bba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:21:13.019340 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec96e74-121c-40de-8fe4-d1f526915bba service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:21:13.029361 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 690/1368] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:21:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:21:15.196695 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4806f8-10e0-4b1d-bfaa-5279d1fc804d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:15.197249 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4806f8-10e0-4b1d-bfaa-5279d1fc804d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:15.197249 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4806f8-10e0-4b1d-bfaa-5279d1fc804d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:15.198695 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 679/1369] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:21:15.204063 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaceafbf-a025-4c70-b920-33ce7bb4b3bc None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:15.204496 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaceafbf-a025-4c70-b920-33ce7bb4b3bc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:15.204630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaceafbf-a025-4c70-b920-33ce7bb4b3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:15.225032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-aaceafbf-a025-4c70-b920-33ce7bb4b3bc None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:15.226582 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-aaceafbf-a025-4c70-b920-33ce7bb4b3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:15.235107 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 691/1370] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:21:15.290330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4806f8-10e0-4b1d-bfaa-5279d1fc804d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:21:15.306278 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4806f8-10e0-4b1d-bfaa-5279d1fc804d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:21:15.312146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eef123-7dcd-40cc-94c3-217fe16f92a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:21:15.312895 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eef123-7dcd-40cc-94c3-217fe16f92a9 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:15.313281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eef123-7dcd-40cc-94c3-217fe16f92a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:15.313696 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eef123-7dcd-40cc-94c3-217fe16f92a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:15.324008 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eef123-7dcd-40cc-94c3-217fe16f92a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:21:15.325799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eef123-7dcd-40cc-94c3-217fe16f92a9 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:21:15.336825 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 680/1371] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:21:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:21:15.550133 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989096f2-eb4a-472f-8d7f-b6759cd69615 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:15.550767 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989096f2-eb4a-472f-8d7f-b6759cd69615 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:15.551233 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989096f2-eb4a-472f-8d7f-b6759cd69615 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:15.552858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 692/1372] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:21:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:21:15.558808 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b91b45-921b-4821-ad18-20c45efcdd0e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:15.559099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b91b45-921b-4821-ad18-20c45efcdd0e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:15.559381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b91b45-921b-4821-ad18-20c45efcdd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:15.581847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-59b91b45-921b-4821-ad18-20c45efcdd0e None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:15.583085 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-59b91b45-921b-4821-ad18-20c45efcdd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:15.591604 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 681/1373] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:21:15.901515 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f317dc-0c7d-467c-a674-95246c8459c8 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:15.902078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f317dc-0c7d-467c-a674-95246c8459c8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:15.902573 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f317dc-0c7d-467c-a674-95246c8459c8 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:15.904091 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 693/1374] 10.222.0.33 () {54 vars in 963 bytes} [Thu Jun 30 18:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:21:15.909589 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fcb6e-9261-4041-875e-39c95d092d90 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:15.909901 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fcb6e-9261-4041-875e-39c95d092d90 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:15.910182 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fcb6e-9261-4041-875e-39c95d092d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:15.961934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-886fcb6e-9261-4041-875e-39c95d092d90 None None] MFA Rules not processed for user `92c70ce830ea4dd981716d622b16d595`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:15.963197 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-886fcb6e-9261-4041-875e-39c95d092d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:16.029533 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 682/1375] 10.222.0.33 () {58 vars in 1060 bytes} [Thu Jun 30 18:21:15 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:21:16.037931 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f317dc-0c7d-467c-a674-95246c8459c8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:21:16.049085 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f317dc-0c7d-467c-a674-95246c8459c8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:21:16.052938 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146a32d-3224-46df-a80e-a1b3ba84d58c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:21:16.053727 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146a32d-3224-46df-a80e-a1b3ba84d58c service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:16.054084 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146a32d-3224-46df-a80e-a1b3ba84d58c service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:16.054434 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146a32d-3224-46df-a80e-a1b3ba84d58c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:16.068464 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3146a32d-3224-46df-a80e-a1b3ba84d58c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:21:16.070068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3146a32d-3224-46df-a80e-a1b3ba84d58c service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:21:16.080174 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 694/1376] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:21:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:21:17.621083 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1e3ad-667c-433c-8d8e-0a78ce90bca1 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:17.621688 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1e3ad-667c-433c-8d8e-0a78ce90bca1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:17.621688 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1e3ad-667c-433c-8d8e-0a78ce90bca1 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:17.623029 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 683/1377] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:21:17.630245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039461d-f74d-465f-9a29-c2f9ad8ad66d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:17.630245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039461d-f74d-465f-9a29-c2f9ad8ad66d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:17.630965 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039461d-f74d-465f-9a29-c2f9ad8ad66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:17.649817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6039461d-f74d-465f-9a29-c2f9ad8ad66d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:17.651106 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6039461d-f74d-465f-9a29-c2f9ad8ad66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:17.660076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 695/1378] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:21:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:21:17.705114 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1e3ad-667c-433c-8d8e-0a78ce90bca1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:21:17.721609 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1e3ad-667c-433c-8d8e-0a78ce90bca1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:21:17.727068 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93876a00-1645-4f4b-a185-34ac303b84b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:21:17.727709 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93876a00-1645-4f4b-a185-34ac303b84b8 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:17.727997 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93876a00-1645-4f4b-a185-34ac303b84b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:17.728279 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93876a00-1645-4f4b-a185-34ac303b84b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:17.737043 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93876a00-1645-4f4b-a185-34ac303b84b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:21:17.738865 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93876a00-1645-4f4b-a185-34ac303b84b8 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:21:17.743096 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:21:17.750315 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 684/1379] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:21:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:21:19.870040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a439a94-b82e-46dd-a8a7-1f6ef63bfb4c None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:19.870582 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a439a94-b82e-46dd-a8a7-1f6ef63bfb4c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:19.870582 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a439a94-b82e-46dd-a8a7-1f6ef63bfb4c None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:19.872150 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 696/1380] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:21:19.877278 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad4863-cb8c-4fd8-ac15-5fbfb96f51e9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:19.879343 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad4863-cb8c-4fd8-ac15-5fbfb96f51e9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:19.879927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad4863-cb8c-4fd8-ac15-5fbfb96f51e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:19.903825 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-17ad4863-cb8c-4fd8-ac15-5fbfb96f51e9 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:19.906134 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-17ad4863-cb8c-4fd8-ac15-5fbfb96f51e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:19.918877 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 685/1381] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:21:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:21:19.970282 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a439a94-b82e-46dd-a8a7-1f6ef63bfb4c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:21:19.990348 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a439a94-b82e-46dd-a8a7-1f6ef63bfb4c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:21:20.000275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40849f4-133e-4a26-b305-823dd7edad33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:21:20.000937 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40849f4-133e-4a26-b305-823dd7edad33 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:20.001229 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40849f4-133e-4a26-b305-823dd7edad33 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:20.001515 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40849f4-133e-4a26-b305-823dd7edad33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:20.012787 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40849f4-133e-4a26-b305-823dd7edad33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:21:20.014439 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40849f4-133e-4a26-b305-823dd7edad33 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:21:20.026772 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 697/1382] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:21:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:21:22.150285 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f6804-672d-4fdc-8bef-22323c849f32 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:22.150793 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f6804-672d-4fdc-8bef-22323c849f32 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:22.150793 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f6804-672d-4fdc-8bef-22323c849f32 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:22.152101 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 686/1383] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:21:22.157465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52364619-e45c-47c3-a26d-301a137bd73c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:22.157915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52364619-e45c-47c3-a26d-301a137bd73c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:22.158060 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52364619-e45c-47c3-a26d-301a137bd73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:22.177359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-52364619-e45c-47c3-a26d-301a137bd73c None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:22.178592 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-52364619-e45c-47c3-a26d-301a137bd73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:22.186555 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 698/1384] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:21:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:21:22.228886 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f6804-672d-4fdc-8bef-22323c849f32 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:21:22.245030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f6804-672d-4fdc-8bef-22323c849f32 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:21:22.250734 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572349a4-cb7f-4691-8240-ede93041efa4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:21:22.251403 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572349a4-cb7f-4691-8240-ede93041efa4 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:22.251742 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572349a4-cb7f-4691-8240-ede93041efa4 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:22.252028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572349a4-cb7f-4691-8240-ede93041efa4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:22.261189 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572349a4-cb7f-4691-8240-ede93041efa4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:21:22.262831 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572349a4-cb7f-4691-8240-ede93041efa4 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:21:22.273223 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 687/1385] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:21:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:21:24.355931 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fec33-9fb4-4330-a16f-b098d8e3f378 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:24.356439 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fec33-9fb4-4330-a16f-b098d8e3f378 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:24.356439 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fec33-9fb4-4330-a16f-b098d8e3f378 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:24.357825 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 699/1386] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:21:24.363908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fdd93-1719-4269-9a20-6d4d8d80196b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:24.364381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fdd93-1719-4269-9a20-6d4d8d80196b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:24.364477 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fdd93-1719-4269-9a20-6d4d8d80196b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:24.382354 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5b5fdd93-1719-4269-9a20-6d4d8d80196b None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:24.383567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5b5fdd93-1719-4269-9a20-6d4d8d80196b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:24.391812 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 688/1387] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:21:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:21:24.437417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fec33-9fb4-4330-a16f-b098d8e3f378 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:21:24.453979 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fec33-9fb4-4330-a16f-b098d8e3f378 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:21:24.460821 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2d856-dcd9-48b3-ad06-99f8aa02cc25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:21:24.461710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2d856-dcd9-48b3-ad06-99f8aa02cc25 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:24.462070 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2d856-dcd9-48b3-ad06-99f8aa02cc25 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:24.462428 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2d856-dcd9-48b3-ad06-99f8aa02cc25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:24.473638 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d2d856-dcd9-48b3-ad06-99f8aa02cc25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:21:24.475943 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d2d856-dcd9-48b3-ad06-99f8aa02cc25 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:21:24.488262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 700/1388] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:21:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:21:26.882901 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abefbfcc-7493-4ed2-b731-09a34f2f3836 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:26.883642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abefbfcc-7493-4ed2-b731-09a34f2f3836 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:26.883642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abefbfcc-7493-4ed2-b731-09a34f2f3836 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:26.884849 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 689/1389] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:21:26.891916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594176f-6112-4ff4-96f4-81ddf1ba7329 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:26.891916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594176f-6112-4ff4-96f4-81ddf1ba7329 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:26.892732 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594176f-6112-4ff4-96f4-81ddf1ba7329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:26.915653 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b594176f-6112-4ff4-96f4-81ddf1ba7329 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:26.916874 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b594176f-6112-4ff4-96f4-81ddf1ba7329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:26.925867 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 701/1390] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:21:26.972074 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abefbfcc-7493-4ed2-b731-09a34f2f3836 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:21:26.987407 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abefbfcc-7493-4ed2-b731-09a34f2f3836 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:21:26.993256 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb178621-2675-4838-ac1a-9add4e061ad5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:21:26.993937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb178621-2675-4838-ac1a-9add4e061ad5 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:26.994224 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb178621-2675-4838-ac1a-9add4e061ad5 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:26.994506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb178621-2675-4838-ac1a-9add4e061ad5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:27.004271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb178621-2675-4838-ac1a-9add4e061ad5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:21:27.005911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb178621-2675-4838-ac1a-9add4e061ad5 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:21:27.015863 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 690/1391] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:21:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:21:29.234445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa948a-ad17-41b5-a047-70bfc2c9eec8 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:29.235008 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa948a-ad17-41b5-a047-70bfc2c9eec8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:29.235008 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa948a-ad17-41b5-a047-70bfc2c9eec8 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:29.236432 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 702/1392] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:21:29.242573 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cbb01-b4e9-4d5c-8f88-547a9e9c3127 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:29.243055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cbb01-b4e9-4d5c-8f88-547a9e9c3127 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:29.243159 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cbb01-b4e9-4d5c-8f88-547a9e9c3127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:29.262500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c11cbb01-b4e9-4d5c-8f88-547a9e9c3127 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:29.264025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c11cbb01-b4e9-4d5c-8f88-547a9e9c3127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:29.275214 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 691/1393] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:21:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:21:29.321308 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa948a-ad17-41b5-a047-70bfc2c9eec8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:21:29.339589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa948a-ad17-41b5-a047-70bfc2c9eec8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:21:29.346892 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14051a71-e31b-4384-82dd-2623ecaaf225 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:21:29.347889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14051a71-e31b-4384-82dd-2623ecaaf225 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:29.348311 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14051a71-e31b-4384-82dd-2623ecaaf225 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:29.348602 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14051a71-e31b-4384-82dd-2623ecaaf225 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:29.359801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14051a71-e31b-4384-82dd-2623ecaaf225 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:21:29.362886 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14051a71-e31b-4384-82dd-2623ecaaf225 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:21:29.375593 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 703/1394] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:21:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:21:31.473445 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0c466-6546-4d2e-a73f-743b7a34a7da None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:31.474015 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0c466-6546-4d2e-a73f-743b7a34a7da None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:31.474015 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0c466-6546-4d2e-a73f-743b7a34a7da None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:31.475381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 692/1395] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:21:31.480769 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41293a-50fb-45fc-82fa-708fac6036e2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:31.481203 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41293a-50fb-45fc-82fa-708fac6036e2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:31.481386 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41293a-50fb-45fc-82fa-708fac6036e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:31.500289 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5f41293a-50fb-45fc-82fa-708fac6036e2 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:31.501540 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5f41293a-50fb-45fc-82fa-708fac6036e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:31.510632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 704/1396] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:21:31.555157 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0c466-6546-4d2e-a73f-743b7a34a7da None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:21:31.570184 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0c466-6546-4d2e-a73f-743b7a34a7da None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:21:31.575625 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c5085-abf5-4197-b127-85a1d4ff81d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:21:31.576256 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c5085-abf5-4197-b127-85a1d4ff81d0 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:31.576536 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c5085-abf5-4197-b127-85a1d4ff81d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:31.576809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c5085-abf5-4197-b127-85a1d4ff81d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:31.585956 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c5085-abf5-4197-b127-85a1d4ff81d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:21:31.587373 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c5085-abf5-4197-b127-85a1d4ff81d0 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:21:31.597106 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 693/1397] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:21:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:21:31.822440 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d864afc-e5e2-47b7-967d-3d822336e471 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:31.822844 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d864afc-e5e2-47b7-967d-3d822336e471 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:31.823134 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d864afc-e5e2-47b7-967d-3d822336e471 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:31.824561 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 705/1398] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:21:31.830667 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a1000-b620-45d3-bdac-532050ea3c22 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:31.830963 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a1000-b620-45d3-bdac-532050ea3c22 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:31.831246 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a1000-b620-45d3-bdac-532050ea3c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:31.849878 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-868a1000-b620-45d3-bdac-532050ea3c22 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:31.851075 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-868a1000-b620-45d3-bdac-532050ea3c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:31.859032 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 694/1399] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:21:31.917485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9633751-5871-4446-a620-eed4cfdadc9a None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:31.918098 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9633751-5871-4446-a620-eed4cfdadc9a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:31.918469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9633751-5871-4446-a620-eed4cfdadc9a None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:31.919869 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 706/1400] 10.222.0.33 () {54 vars in 962 bytes} [Thu Jun 30 18:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:21:31.925727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165561d-0905-47eb-a875-aeb1b2250bf8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:31.926021 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165561d-0905-47eb-a875-aeb1b2250bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:31.926301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165561d-0905-47eb-a875-aeb1b2250bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:31.975534 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3165561d-0905-47eb-a875-aeb1b2250bf8 None None] MFA Rules not processed for user `37a9fea702cd405ea9fc14ec2c689648`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:31.976716 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3165561d-0905-47eb-a875-aeb1b2250bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:32.039354 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 695/1401] 10.222.0.33 () {58 vars in 1059 bytes} [Thu Jun 30 18:21:31 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:21:32.047056 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9633751-5871-4446-a620-eed4cfdadc9a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:21:32.058413 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9633751-5871-4446-a620-eed4cfdadc9a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:21:32.062483 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3d90d-5985-467f-8473-7106954ac0a4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37a9fea702cd405ea9fc14ec2c689648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:21:32.063129 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3d90d-5985-467f-8473-7106954ac0a4 service glance] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:32.065913 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3d90d-5985-467f-8473-7106954ac0a4 service glance] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:32.066223 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3d90d-5985-467f-8473-7106954ac0a4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:32.076099 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3d90d-5985-467f-8473-7106954ac0a4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:21:32.077553 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3d90d-5985-467f-8473-7106954ac0a4 service glance] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:21:32.087210 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 707/1402] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:21:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:21:32.821922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b31a1b-deb7-4b0a-a6df-e1523c8daf03 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:32.822403 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b31a1b-deb7-4b0a-a6df-e1523c8daf03 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:32.822629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b31a1b-deb7-4b0a-a6df-e1523c8daf03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:32.872757 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-57b31a1b-deb7-4b0a-a6df-e1523c8daf03 None None] MFA Rules not processed for user `4ddcf6306e704fe9ad5ed8779178ae67`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:32.874017 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-57b31a1b-deb7-4b0a-a6df-e1523c8daf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:32.915446 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:21:32.939834 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 696/1403] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 30 18:21:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:21:32.983452 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6485aa-0e5c-437f-8c56-73987d745809 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:32.983943 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6485aa-0e5c-437f-8c56-73987d745809 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:32.984042 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6485aa-0e5c-437f-8c56-73987d745809 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:32.985392 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 708/1404] 10.222.0.33 () {54 vars in 942 bytes} [Thu Jun 30 18:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:21:32.991996 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce2fed-e89b-44c8-9de5-00b1cc540506 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:32.992292 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce2fed-e89b-44c8-9de5-00b1cc540506 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:32.992577 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce2fed-e89b-44c8-9de5-00b1cc540506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:33.042333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-46ce2fed-e89b-44c8-9de5-00b1cc540506 None None] MFA Rules not processed for user `4bd521b9e93f4b3b8af64d46f36c019e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:33.043548 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-46ce2fed-e89b-44c8-9de5-00b1cc540506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:33.119903 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 697/1405] 10.222.0.33 () {58 vars in 1039 bytes} [Thu Jun 30 18:21:32 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:21:33.129484 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6485aa-0e5c-437f-8c56-73987d745809 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:21:33.136869 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=61 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:21:33.144254 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6485aa-0e5c-437f-8c56-73987d745809 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:21:33.149604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f88183-9de4-4dcc-98e8-6bc9c0b2bc36 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bd521b9e93f4b3b8af64d46f36c019e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:21:33.150251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f88183-9de4-4dcc-98e8-6bc9c0b2bc36 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:33.150530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f88183-9de4-4dcc-98e8-6bc9c0b2bc36 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:33.150814 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f88183-9de4-4dcc-98e8-6bc9c0b2bc36 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:33.160083 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f88183-9de4-4dcc-98e8-6bc9c0b2bc36 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:21:33.162129 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f88183-9de4-4dcc-98e8-6bc9c0b2bc36 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:21:33.183867 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 709/1406] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 30 18:21:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 18:21:33.718272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc11d0c-0c3a-433d-8dc4-04757469dbcb None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:33.718558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc11d0c-0c3a-433d-8dc4-04757469dbcb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:33.718833 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc11d0c-0c3a-433d-8dc4-04757469dbcb None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:33.720244 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 698/1407] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:21:33.725482 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed2191-d7f7-4228-9017-ba9665675143 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:33.725810 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed2191-d7f7-4228-9017-ba9665675143 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:33.726121 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed2191-d7f7-4228-9017-ba9665675143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:33.746060 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-06ed2191-d7f7-4228-9017-ba9665675143 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:33.747454 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-06ed2191-d7f7-4228-9017-ba9665675143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:33.760515 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 710/1408] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:21:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:21:33.807296 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc11d0c-0c3a-433d-8dc4-04757469dbcb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:21:33.824174 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc11d0c-0c3a-433d-8dc4-04757469dbcb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:21:33.832793 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a7926-1bde-44e6-9c4d-ca162d3ffeba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:21:33.832793 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a7926-1bde-44e6-9c4d-ca162d3ffeba service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:33.832793 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a7926-1bde-44e6-9c4d-ca162d3ffeba service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:33.832793 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a7926-1bde-44e6-9c4d-ca162d3ffeba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:33.844199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a7926-1bde-44e6-9c4d-ca162d3ffeba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:21:33.846642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a7926-1bde-44e6-9c4d-ca162d3ffeba service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:21:33.856328 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 699/1409] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:21:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:21:43.740716 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=8 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:21:43.851974 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=59 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:21:55.708680 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6055fe-2c44-459c-8222-ea37f4682df5 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:55.709256 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6055fe-2c44-459c-8222-ea37f4682df5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:55.709256 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6055fe-2c44-459c-8222-ea37f4682df5 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:55.710689 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 711/1410] 10.222.0.33 () {54 vars in 918 bytes} [Thu Jun 30 18:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:21:55.715668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0f265-b679-451a-bb62-6ff331a4c95f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:55.716079 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0f265-b679-451a-bb62-6ff331a4c95f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:55.716220 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0f265-b679-451a-bb62-6ff331a4c95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:55.735895 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e7c0f265-b679-451a-bb62-6ff331a4c95f None None] MFA Rules not processed for user `49325c1733174e0c98b3475a10c9b5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:55.737103 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e7c0f265-b679-451a-bb62-6ff331a4c95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:55.745750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 700/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Thu Jun 30 18:21:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:21:55.760441 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1a69a-219f-45a7-9d9a-0b0101ddaef0 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:55.760733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1a69a-219f-45a7-9d9a-0b0101ddaef0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:55.761014 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1a69a-219f-45a7-9d9a-0b0101ddaef0 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:55.762423 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 712/1412] 10.222.0.33 () {54 vars in 918 bytes} [Thu Jun 30 18:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:21:55.768071 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789f34f-c46c-4066-b83c-208d0228569c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:55.768365 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789f34f-c46c-4066-b83c-208d0228569c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:55.768650 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789f34f-c46c-4066-b83c-208d0228569c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:55.786901 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7789f34f-c46c-4066-b83c-208d0228569c None None] MFA Rules not processed for user `b8751a4e92174d18b7b138392d2579a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:55.788128 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7789f34f-c46c-4066-b83c-208d0228569c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:55.797553 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 701/1413] 10.222.0.33 () {58 vars in 1015 bytes} [Thu Jun 30 18:21:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:21:58.362747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3a711-a2bd-4cf7-8ca4-8e7c4863d1ee None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:58.363289 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3a711-a2bd-4cf7-8ca4-8e7c4863d1ee None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:58.363289 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3a711-a2bd-4cf7-8ca4-8e7c4863d1ee None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:58.364589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 18:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:21:58.369950 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab783f2-01e6-470c-8327-b485960ea378 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:58.370388 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab783f2-01e6-470c-8327-b485960ea378 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:58.370516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab783f2-01e6-470c-8327-b485960ea378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:58.390563 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8ab783f2-01e6-470c-8327-b485960ea378 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:58.391772 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8ab783f2-01e6-470c-8327-b485960ea378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:58.399611 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 18:21:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:21:58.443143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3a711-a2bd-4cf7-8ca4-8e7c4863d1ee None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:21:58.459056 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3a711-a2bd-4cf7-8ca4-8e7c4863d1ee None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:21:58.464425 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaae84a-b12e-40b3-8d8e-7e68fcc293a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:21:58.465058 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaae84a-b12e-40b3-8d8e-7e68fcc293a6 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:58.465343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaae84a-b12e-40b3-8d8e-7e68fcc293a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:58.465653 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaae84a-b12e-40b3-8d8e-7e68fcc293a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:58.474803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaae84a-b12e-40b3-8d8e-7e68fcc293a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:21:58.476389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaae84a-b12e-40b3-8d8e-7e68fcc293a6 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:21:58.486195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 714/1416] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:21:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:21:59.385066 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdce87c-c342-4269-b3c9-d69ced9d2578 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:59.385656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdce87c-c342-4269-b3c9-d69ced9d2578 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:59.385656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdce87c-c342-4269-b3c9-d69ced9d2578 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:59.387004 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 18:21:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:21:59.392652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdef04-b751-45a9-8632-3692b5fe4433 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:59.393069 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdef04-b751-45a9-8632-3692b5fe4433 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:59.393221 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdef04-b751-45a9-8632-3692b5fe4433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:59.411603 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0dfdef04-b751-45a9-8632-3692b5fe4433 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:21:59.412821 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0dfdef04-b751-45a9-8632-3692b5fe4433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:59.420862 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 18:21:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:21:59.572144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdce87c-c342-4269-b3c9-d69ced9d2578 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:21:59.587149 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdce87c-c342-4269-b3c9-d69ced9d2578 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:21:59.592812 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a36e62-2273-4c56-9824-f92188dfdbfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:21:59.593455 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a36e62-2273-4c56-9824-f92188dfdbfb service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:59.593780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a36e62-2273-4c56-9824-f92188dfdbfb service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:59.594065 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a36e62-2273-4c56-9824-f92188dfdbfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:59.602541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a36e62-2273-4c56-9824-f92188dfdbfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:21:59.604143 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a36e62-2273-4c56-9824-f92188dfdbfb service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:21:59.614093 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 704/1419] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:21:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:22:00.505203 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9d74a-0267-4628-b1b6-366d9693b0e1 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:00.505774 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9d74a-0267-4628-b1b6-366d9693b0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:00.505774 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9d74a-0267-4628-b1b6-366d9693b0e1 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:00.507150 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 18:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:22:00.512092 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270939bb-1349-438c-85f4-84686cde68dc None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:00.512504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270939bb-1349-438c-85f4-84686cde68dc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:00.512645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270939bb-1349-438c-85f4-84686cde68dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:00.531036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-270939bb-1349-438c-85f4-84686cde68dc None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:00.532227 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-270939bb-1349-438c-85f4-84686cde68dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:00.540308 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 18:22:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:22:00.695364 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9d74a-0267-4628-b1b6-366d9693b0e1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:00.710263 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9d74a-0267-4628-b1b6-366d9693b0e1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:00.715912 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9bf8c6-ddc0-45c5-8574-b44c3f4d484b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:00.717179 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bf8c6-ddc0-45c5-8574-b44c3f4d484b service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:00.717627 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bf8c6-ddc0-45c5-8574-b44c3f4d484b service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:00.718015 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bf8c6-ddc0-45c5-8574-b44c3f4d484b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:00.726338 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9bf8c6-ddc0-45c5-8574-b44c3f4d484b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:00.727937 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9bf8c6-ddc0-45c5-8574-b44c3f4d484b service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:00.737810 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 717/1422] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:22:01.604995 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c44b2-bbc2-4da2-8e3f-eac09606e821 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:01.605623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c44b2-bbc2-4da2-8e3f-eac09606e821 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:01.605623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c44b2-bbc2-4da2-8e3f-eac09606e821 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:01.606959 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 18:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:22:01.612457 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059627f3-1b09-49fb-9144-fef1c31ab671 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:01.612922 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059627f3-1b09-49fb-9144-fef1c31ab671 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:01.613127 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059627f3-1b09-49fb-9144-fef1c31ab671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:01.633694 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-059627f3-1b09-49fb-9144-fef1c31ab671 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:01.634931 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-059627f3-1b09-49fb-9144-fef1c31ab671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:01.646455 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 18:22:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:22:01.804745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c44b2-bbc2-4da2-8e3f-eac09606e821 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:01.820247 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c44b2-bbc2-4da2-8e3f-eac09606e821 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:01.825656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e1d5f-98d0-4f51-8c3d-141d53ab6d02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:01.826862 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e1d5f-98d0-4f51-8c3d-141d53ab6d02 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:01.827152 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e1d5f-98d0-4f51-8c3d-141d53ab6d02 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:01.827435 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e1d5f-98d0-4f51-8c3d-141d53ab6d02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:01.835618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e1d5f-98d0-4f51-8c3d-141d53ab6d02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:01.837206 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e1d5f-98d0-4f51-8c3d-141d53ab6d02 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:01.847690 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 707/1425] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:22:02.726016 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188a01f-cbba-447e-bb2f-3cc20125ae68 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:02.726577 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188a01f-cbba-447e-bb2f-3cc20125ae68 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:02.726577 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188a01f-cbba-447e-bb2f-3cc20125ae68 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:02.727947 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 18:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:22:02.732947 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d851bf3-0b61-4264-9d29-eb4b23f627d8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:02.733377 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d851bf3-0b61-4264-9d29-eb4b23f627d8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:02.733479 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d851bf3-0b61-4264-9d29-eb4b23f627d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:02.751760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8d851bf3-0b61-4264-9d29-eb4b23f627d8 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:02.752958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8d851bf3-0b61-4264-9d29-eb4b23f627d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:02.761180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 18:22:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:22:02.911778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188a01f-cbba-447e-bb2f-3cc20125ae68 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:02.927287 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188a01f-cbba-447e-bb2f-3cc20125ae68 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:02.932697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369b369-9c02-44d0-b579-6e0da115cf27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:02.933328 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369b369-9c02-44d0-b579-6e0da115cf27 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:02.933639 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369b369-9c02-44d0-b579-6e0da115cf27 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:02.933925 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369b369-9c02-44d0-b579-6e0da115cf27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:02.942773 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369b369-9c02-44d0-b579-6e0da115cf27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:02.944351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369b369-9c02-44d0-b579-6e0da115cf27 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:02.954781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 720/1428] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:22:03.757362 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84821b-da07-478b-8653-c44e07899255 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:03.758268 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84821b-da07-478b-8653-c44e07899255 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:03.758562 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84821b-da07-478b-8653-c44e07899255 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:03.759984 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 18:22:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:22:03.765831 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09aaca9-3150-42f0-9847-327582edf5b3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:03.766290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09aaca9-3150-42f0-9847-327582edf5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:03.766570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09aaca9-3150-42f0-9847-327582edf5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:03.787207 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e09aaca9-3150-42f0-9847-327582edf5b3 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:03.788416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e09aaca9-3150-42f0-9847-327582edf5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:03.796678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 18:22:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:22:03.950106 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84821b-da07-478b-8653-c44e07899255 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:03.965664 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84821b-da07-478b-8653-c44e07899255 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:03.971789 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a8df2-3721-4a88-92ec-028e2dd6553d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:03.972455 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a8df2-3721-4a88-92ec-028e2dd6553d service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:03.972739 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a8df2-3721-4a88-92ec-028e2dd6553d service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:03.973014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a8df2-3721-4a88-92ec-028e2dd6553d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:03.981233 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a8df2-3721-4a88-92ec-028e2dd6553d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:03.983065 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a8df2-3721-4a88-92ec-028e2dd6553d service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:03.992429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 710/1431] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:22:04.789036 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fcc0ac-56fc-4d99-af4a-39c04cc35005 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:04.789664 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fcc0ac-56fc-4d99-af4a-39c04cc35005 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:04.789664 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fcc0ac-56fc-4d99-af4a-39c04cc35005 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:04.791048 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 18:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:22:04.796318 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373923dc-3c38-45e0-8d6e-f51cb3a42c18 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:04.796846 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373923dc-3c38-45e0-8d6e-f51cb3a42c18 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:04.796954 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373923dc-3c38-45e0-8d6e-f51cb3a42c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:04.815607 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-373923dc-3c38-45e0-8d6e-f51cb3a42c18 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:04.816816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-373923dc-3c38-45e0-8d6e-f51cb3a42c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:04.824708 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 18:22:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:22:04.975933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fcc0ac-56fc-4d99-af4a-39c04cc35005 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:04.995765 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fcc0ac-56fc-4d99-af4a-39c04cc35005 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:05.001989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b367e31-c349-4240-8332-1bbaf1dc2c63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:05.002634 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b367e31-c349-4240-8332-1bbaf1dc2c63 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:05.002922 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b367e31-c349-4240-8332-1bbaf1dc2c63 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:05.003203 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b367e31-c349-4240-8332-1bbaf1dc2c63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:05.011607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b367e31-c349-4240-8332-1bbaf1dc2c63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:05.013318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b367e31-c349-4240-8332-1bbaf1dc2c63 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:05.023117 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 723/1434] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:22:05.840516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55a866-db39-43ed-8752-81f7d32e07c7 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:05.841617 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55a866-db39-43ed-8752-81f7d32e07c7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:05.842095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55a866-db39-43ed-8752-81f7d32e07c7 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:05.843501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 18:22:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:22:05.848736 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117e5c8-a32a-437f-afb2-78e6d468e17f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:05.849198 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117e5c8-a32a-437f-afb2-78e6d468e17f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:05.849305 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117e5c8-a32a-437f-afb2-78e6d468e17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:05.867369 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1117e5c8-a32a-437f-afb2-78e6d468e17f None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:05.868561 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1117e5c8-a32a-437f-afb2-78e6d468e17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:05.876561 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 18:22:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:22:06.029357 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55a866-db39-43ed-8752-81f7d32e07c7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:06.043721 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55a866-db39-43ed-8752-81f7d32e07c7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:06.049513 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f716f-0c35-4189-a148-e9e558cbf845 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:06.050183 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f716f-0c35-4189-a148-e9e558cbf845 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:06.050468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f716f-0c35-4189-a148-e9e558cbf845 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:06.050746 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f716f-0c35-4189-a148-e9e558cbf845 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:06.059319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f716f-0c35-4189-a148-e9e558cbf845 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:06.060994 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f716f-0c35-4189-a148-e9e558cbf845 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:06.071180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 713/1437] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:22:06.884893 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141d46e-0caa-406f-a168-a21c98648fed None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:06.885477 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141d46e-0caa-406f-a168-a21c98648fed None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:06.885477 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141d46e-0caa-406f-a168-a21c98648fed None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:06.886952 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 18:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:22:06.892228 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fa8db-d14b-40cf-9f7d-d6afa4296fbf None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:06.892674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fa8db-d14b-40cf-9f7d-d6afa4296fbf None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:06.892781 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fa8db-d14b-40cf-9f7d-d6afa4296fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:06.911017 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-086fa8db-d14b-40cf-9f7d-d6afa4296fbf None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:06.912235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-086fa8db-d14b-40cf-9f7d-d6afa4296fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:06.921259 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 18:22:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:22:07.073559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3141d46e-0caa-406f-a168-a21c98648fed None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:07.088519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3141d46e-0caa-406f-a168-a21c98648fed None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:07.094901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1ffe1-8478-4c29-9d2a-63c7d3e76271 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:07.095539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1ffe1-8478-4c29-9d2a-63c7d3e76271 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:07.095861 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1ffe1-8478-4c29-9d2a-63c7d3e76271 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:07.096140 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1ffe1-8478-4c29-9d2a-63c7d3e76271 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:07.104406 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1ffe1-8478-4c29-9d2a-63c7d3e76271 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:07.106036 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1ffe1-8478-4c29-9d2a-63c7d3e76271 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:07.116380 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 726/1440] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:22:09.084357 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ab650-9316-42c2-9661-6eca688baf98 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:09.084917 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ab650-9316-42c2-9661-6eca688baf98 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:09.085024 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ab650-9316-42c2-9661-6eca688baf98 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:09.086407 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 18:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:22:09.092167 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939415f-eda7-4084-b1e5-f815142cf43b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:09.092578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939415f-eda7-4084-b1e5-f815142cf43b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:09.092732 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939415f-eda7-4084-b1e5-f815142cf43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:09.111155 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f939415f-eda7-4084-b1e5-f815142cf43b None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:09.112359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f939415f-eda7-4084-b1e5-f815142cf43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:09.120353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 18:22:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:22:09.160183 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ab650-9316-42c2-9661-6eca688baf98 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:09.171547 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ab650-9316-42c2-9661-6eca688baf98 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:09.178597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25795dab-6d9b-434b-9504-f637764b4826 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:09.178597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25795dab-6d9b-434b-9504-f637764b4826 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:09.178597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25795dab-6d9b-434b-9504-f637764b4826 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:09.178597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25795dab-6d9b-434b-9504-f637764b4826 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:09.187777 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25795dab-6d9b-434b-9504-f637764b4826 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:09.189476 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25795dab-6d9b-434b-9504-f637764b4826 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:09.199548 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 716/1443] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:22:11.422158 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dfd73-4f44-4658-86c8-358b593eb816 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:11.422717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dfd73-4f44-4658-86c8-358b593eb816 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:11.422717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dfd73-4f44-4658-86c8-358b593eb816 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:11.424114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 18:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:22:11.429505 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb58e11-135d-4a59-8008-f75db4500cd1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:11.429953 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb58e11-135d-4a59-8008-f75db4500cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:11.430099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb58e11-135d-4a59-8008-f75db4500cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:11.449249 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-bcb58e11-135d-4a59-8008-f75db4500cd1 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:11.451816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bcb58e11-135d-4a59-8008-f75db4500cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:11.460778 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 18:22:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:22:11.500646 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146dfd73-4f44-4658-86c8-358b593eb816 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:11.511291 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146dfd73-4f44-4658-86c8-358b593eb816 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:11.515106 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7d938-d34d-43c4-94d7-62cc6fa9c294 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:11.515741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7d938-d34d-43c4-94d7-62cc6fa9c294 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:11.516031 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7d938-d34d-43c4-94d7-62cc6fa9c294 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:11.516313 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7d938-d34d-43c4-94d7-62cc6fa9c294 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:11.525161 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f7d938-d34d-43c4-94d7-62cc6fa9c294 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:11.526626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f7d938-d34d-43c4-94d7-62cc6fa9c294 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:11.536911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 729/1446] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:22:13.941840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d08da-c135-4b4b-9427-c4af9fff591c None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:13.942407 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d08da-c135-4b4b-9427-c4af9fff591c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:13.942407 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d08da-c135-4b4b-9427-c4af9fff591c None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:13.943778 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 18:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:22:13.949067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be6ef4-e332-49d9-a8ab-453f434ee21e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:13.950090 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be6ef4-e332-49d9-a8ab-453f434ee21e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:13.950851 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be6ef4-e332-49d9-a8ab-453f434ee21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:13.968748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-55be6ef4-e332-49d9-a8ab-453f434ee21e None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:13.969988 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-55be6ef4-e332-49d9-a8ab-453f434ee21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:13.977910 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 18:22:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:22:14.018367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d08da-c135-4b4b-9427-c4af9fff591c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:14.029205 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d08da-c135-4b4b-9427-c4af9fff591c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:14.032685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29809bb-5285-4c20-bdff-8b864b1e1031 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:14.033314 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29809bb-5285-4c20-bdff-8b864b1e1031 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:14.033626 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29809bb-5285-4c20-bdff-8b864b1e1031 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:14.033960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29809bb-5285-4c20-bdff-8b864b1e1031 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:14.042935 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29809bb-5285-4c20-bdff-8b864b1e1031 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:14.044482 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29809bb-5285-4c20-bdff-8b864b1e1031 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:14.053998 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 719/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:22:16.266240 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eaf4a1-dc2e-4ca9-b39c-cdc1ae63e350 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:16.266776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eaf4a1-dc2e-4ca9-b39c-cdc1ae63e350 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:16.266880 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eaf4a1-dc2e-4ca9-b39c-cdc1ae63e350 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:16.268191 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 18:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:22:16.273551 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cce2b4-6194-488a-b86d-4a6b3285e495 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:16.274021 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cce2b4-6194-488a-b86d-4a6b3285e495 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:16.274119 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cce2b4-6194-488a-b86d-4a6b3285e495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:16.291965 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f9cce2b4-6194-488a-b86d-4a6b3285e495 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:16.293797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f9cce2b4-6194-488a-b86d-4a6b3285e495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:16.301484 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 18:22:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:22:16.342379 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eaf4a1-dc2e-4ca9-b39c-cdc1ae63e350 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:16.353374 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eaf4a1-dc2e-4ca9-b39c-cdc1ae63e350 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:16.358201 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667ef34-47fb-41c7-80a8-e0d56d0a7f63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:16.358873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667ef34-47fb-41c7-80a8-e0d56d0a7f63 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:16.359163 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667ef34-47fb-41c7-80a8-e0d56d0a7f63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:16.359447 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667ef34-47fb-41c7-80a8-e0d56d0a7f63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:16.368454 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667ef34-47fb-41c7-80a8-e0d56d0a7f63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:16.369914 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667ef34-47fb-41c7-80a8-e0d56d0a7f63 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:16.379324 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 732/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:22:18.690881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80988fa8-7854-4c9f-a870-0f620c259eac None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:18.691412 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80988fa8-7854-4c9f-a870-0f620c259eac None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:18.691412 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80988fa8-7854-4c9f-a870-0f620c259eac None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:18.692791 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 18:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:22:18.697967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcc99d-7754-4566-811b-ace5f30dfdeb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:18.698355 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcc99d-7754-4566-811b-ace5f30dfdeb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:18.698569 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcc99d-7754-4566-811b-ace5f30dfdeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:18.717379 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4efcc99d-7754-4566-811b-ace5f30dfdeb None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:18.718620 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4efcc99d-7754-4566-811b-ace5f30dfdeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:18.726406 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 18:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:22:18.764097 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80988fa8-7854-4c9f-a870-0f620c259eac None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:18.774529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80988fa8-7854-4c9f-a870-0f620c259eac None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:18.778744 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639dac61-9049-4d6d-ad51-4e59a4ab3277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:18.779418 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dac61-9049-4d6d-ad51-4e59a4ab3277 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:18.779647 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dac61-9049-4d6d-ad51-4e59a4ab3277 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:18.779920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dac61-9049-4d6d-ad51-4e59a4ab3277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:18.788526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639dac61-9049-4d6d-ad51-4e59a4ab3277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:18.789996 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639dac61-9049-4d6d-ad51-4e59a4ab3277 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:18.799197 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 722/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:22:20.925958 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294e7a1-010e-4ea7-abf4-6c14debbfdc4 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:20.926508 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294e7a1-010e-4ea7-abf4-6c14debbfdc4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:20.926508 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294e7a1-010e-4ea7-abf4-6c14debbfdc4 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:20.927888 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 18:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:22:20.934150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e3d0d-b9dc-452d-be15-85baa79f7098 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:20.934671 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e3d0d-b9dc-452d-be15-85baa79f7098 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:20.934781 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e3d0d-b9dc-452d-be15-85baa79f7098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:20.953038 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b64e3d0d-b9dc-452d-be15-85baa79f7098 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:20.954338 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b64e3d0d-b9dc-452d-be15-85baa79f7098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:20.962180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 18:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:22:21.001468 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294e7a1-010e-4ea7-abf4-6c14debbfdc4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:21.012015 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294e7a1-010e-4ea7-abf4-6c14debbfdc4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:21.015441 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2a635-fc16-4789-94fe-ba28c6bd356c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:21.016071 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2a635-fc16-4789-94fe-ba28c6bd356c service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:21.016359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2a635-fc16-4789-94fe-ba28c6bd356c service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:21.017292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2a635-fc16-4789-94fe-ba28c6bd356c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:21.025754 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2a635-fc16-4789-94fe-ba28c6bd356c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:21.027267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2a635-fc16-4789-94fe-ba28c6bd356c service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:21.036975 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 735/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:22:23.164723 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1061f-4a9a-4813-8b0d-e50678a6ebc2 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:23.165262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1061f-4a9a-4813-8b0d-e50678a6ebc2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:23.165359 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1061f-4a9a-4813-8b0d-e50678a6ebc2 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:23.166758 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 18:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:22:23.173092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e452398-68c6-46d3-9cff-6bf4d055b2b6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:23.173531 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e452398-68c6-46d3-9cff-6bf4d055b2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:23.173703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e452398-68c6-46d3-9cff-6bf4d055b2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:23.192884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0e452398-68c6-46d3-9cff-6bf4d055b2b6 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:23.194280 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0e452398-68c6-46d3-9cff-6bf4d055b2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:23.203302 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 18:22:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:22:23.241973 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf1061f-4a9a-4813-8b0d-e50678a6ebc2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:23.253058 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf1061f-4a9a-4813-8b0d-e50678a6ebc2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:23.256655 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801b132-31d9-4dfc-b42e-346d318357eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:23.257295 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801b132-31d9-4dfc-b42e-346d318357eb service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:23.257589 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801b132-31d9-4dfc-b42e-346d318357eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:23.257880 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801b132-31d9-4dfc-b42e-346d318357eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:23.267823 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c801b132-31d9-4dfc-b42e-346d318357eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:23.269257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c801b132-31d9-4dfc-b42e-346d318357eb service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:23.279270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 725/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:22:25.382305 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed35fc-9694-4206-b9c1-5f461a6ba2f0 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:25.382845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed35fc-9694-4206-b9c1-5f461a6ba2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:25.382845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed35fc-9694-4206-b9c1-5f461a6ba2f0 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:25.384188 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 18:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:22:25.389779 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fba1e4-1170-4ada-97ba-6c60297cec06 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:25.390217 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fba1e4-1170-4ada-97ba-6c60297cec06 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:25.390332 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fba1e4-1170-4ada-97ba-6c60297cec06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:25.396388 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=79 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:22:25.410431 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-21fba1e4-1170-4ada-97ba-6c60297cec06 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:25.411630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-21fba1e4-1170-4ada-97ba-6c60297cec06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:25.420590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 18:22:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:22:25.459279 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed35fc-9694-4206-b9c1-5f461a6ba2f0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:25.466888 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=73 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:22:25.473423 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed35fc-9694-4206-b9c1-5f461a6ba2f0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:25.476987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314461c-8f21-49fb-805c-29854804e954 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:25.477693 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314461c-8f21-49fb-805c-29854804e954 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:25.477986 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314461c-8f21-49fb-805c-29854804e954 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:25.478271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314461c-8f21-49fb-805c-29854804e954 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:25.487881 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9314461c-8f21-49fb-805c-29854804e954 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:25.489315 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9314461c-8f21-49fb-805c-29854804e954 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:25.498812 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 738/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:22:27.600635 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27610aa0-636d-455a-a2fb-4fc66b262191 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:27.601177 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27610aa0-636d-455a-a2fb-4fc66b262191 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:27.601177 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27610aa0-636d-455a-a2fb-4fc66b262191 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:27.602604 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 18:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:22:27.608662 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c262b-996d-4067-b7fe-9b2315ef1b05 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:27.609157 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c262b-996d-4067-b7fe-9b2315ef1b05 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:27.609277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c262b-996d-4067-b7fe-9b2315ef1b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:27.626992 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-618c262b-996d-4067-b7fe-9b2315ef1b05 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:27.628219 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-618c262b-996d-4067-b7fe-9b2315ef1b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:27.636719 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 18:22:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:22:27.674122 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27610aa0-636d-455a-a2fb-4fc66b262191 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:27.685328 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27610aa0-636d-455a-a2fb-4fc66b262191 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:27.689130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54186134-f265-4d53-981d-97a69d990ae4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:27.689796 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54186134-f265-4d53-981d-97a69d990ae4 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:27.690081 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54186134-f265-4d53-981d-97a69d990ae4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:27.690368 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54186134-f265-4d53-981d-97a69d990ae4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:27.699987 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54186134-f265-4d53-981d-97a69d990ae4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:27.701438 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54186134-f265-4d53-981d-97a69d990ae4 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:27.710839 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 728/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:22:29.810066 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca51d3-9e80-483f-aa84-eaeef03b9fc7 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:29.810653 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca51d3-9e80-483f-aa84-eaeef03b9fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:29.810754 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca51d3-9e80-483f-aa84-eaeef03b9fc7 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:29.812083 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 18:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:22:29.817913 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dc074-6f71-43da-8d13-37893a9806e8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:29.818357 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dc074-6f71-43da-8d13-37893a9806e8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:29.818452 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dc074-6f71-43da-8d13-37893a9806e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:29.836420 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2c9dc074-6f71-43da-8d13-37893a9806e8 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:29.837666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2c9dc074-6f71-43da-8d13-37893a9806e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:29.846593 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 18:22:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:22:29.883479 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca51d3-9e80-483f-aa84-eaeef03b9fc7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:29.894824 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca51d3-9e80-483f-aa84-eaeef03b9fc7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:29.898509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d6c81-517c-4326-9967-26e81f7f8a55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:29.899145 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d6c81-517c-4326-9967-26e81f7f8a55 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:29.899432 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d6c81-517c-4326-9967-26e81f7f8a55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:29.899712 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d6c81-517c-4326-9967-26e81f7f8a55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:29.909698 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d6c81-517c-4326-9967-26e81f7f8a55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:29.911165 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d6c81-517c-4326-9967-26e81f7f8a55 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:29.921673 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 741/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:22:39.831211 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=11 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:22:39.916347 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=14 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:22:50.256949 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08d18d-1a3f-4ecf-bc80-3c43eff8d435 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:50.257599 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08d18d-1a3f-4ecf-bc80-3c43eff8d435 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:50.257599 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08d18d-1a3f-4ecf-bc80-3c43eff8d435 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:50.258982 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 730/1471] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:22:50.265345 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353d870-a956-45d7-89ff-bcc8de48af0a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:50.265789 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353d870-a956-45d7-89ff-bcc8de48af0a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:50.265924 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353d870-a956-45d7-89ff-bcc8de48af0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:50.283998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b353d870-a956-45d7-89ff-bcc8de48af0a None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:50.285140 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b353d870-a956-45d7-89ff-bcc8de48af0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:50.293142 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 742/1472] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:22:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:22:50.344292 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388314a4-5ae4-44f1-a09d-f7066ee6a9ed None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:50.344579 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388314a4-5ae4-44f1-a09d-f7066ee6a9ed None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:50.344867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388314a4-5ae4-44f1-a09d-f7066ee6a9ed None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:50.346263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 731/1473] 10.222.0.33 () {54 vars in 962 bytes} [Thu Jun 30 18:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:22:50.351578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91421a-6552-489a-9378-54ef368aac96 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:50.351872 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91421a-6552-489a-9378-54ef368aac96 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:50.352161 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91421a-6552-489a-9378-54ef368aac96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:50.371244 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6a91421a-6552-489a-9378-54ef368aac96 None None] MFA Rules not processed for user `37a9fea702cd405ea9fc14ec2c689648`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:50.372456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6a91421a-6552-489a-9378-54ef368aac96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:50.378917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 743/1474] 10.222.0.33 () {58 vars in 1059 bytes} [Thu Jun 30 18:22:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:22:50.387545 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388314a4-5ae4-44f1-a09d-f7066ee6a9ed None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:50.398416 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388314a4-5ae4-44f1-a09d-f7066ee6a9ed None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:50.403245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0f9a0-2fb7-449c-a950-193991090805 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37a9fea702cd405ea9fc14ec2c689648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:50.403872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0f9a0-2fb7-449c-a950-193991090805 service glance] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:50.404155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0f9a0-2fb7-449c-a950-193991090805 service glance] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:50.404434 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0f9a0-2fb7-449c-a950-193991090805 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:50.413392 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0f9a0-2fb7-449c-a950-193991090805 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:50.414861 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0f9a0-2fb7-449c-a950-193991090805 service glance] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:50.424909 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 732/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:22:52.646967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540c1e8-f3a0-4687-af60-e0730d577a41 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:52.647800 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540c1e8-f3a0-4687-af60-e0730d577a41 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:52.647988 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540c1e8-f3a0-4687-af60-e0730d577a41 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:52.649551 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 744/1476] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:22:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:22:52.655058 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec54778-2a81-4e6b-9aec-274c150ea1ed None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:52.655560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec54778-2a81-4e6b-9aec-274c150ea1ed None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:52.655680 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec54778-2a81-4e6b-9aec-274c150ea1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:52.673663 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-dec54778-2a81-4e6b-9aec-274c150ea1ed None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:52.674873 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dec54778-2a81-4e6b-9aec-274c150ea1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:52.683547 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 733/1477] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:22:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:22:52.707489 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540c1e8-f3a0-4687-af60-e0730d577a41 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:52.720066 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540c1e8-f3a0-4687-af60-e0730d577a41 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:52.724549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507fe3fa-1ba6-4124-84cd-6ca4753978cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:52.725205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507fe3fa-1ba6-4124-84cd-6ca4753978cc admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:52.725495 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507fe3fa-1ba6-4124-84cd-6ca4753978cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:52.725874 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507fe3fa-1ba6-4124-84cd-6ca4753978cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:52.726615 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507fe3fa-1ba6-4124-84cd-6ca4753978cc admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:52.728905 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-507fe3fa-1ba6-4124-84cd-6ca4753978cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 18:22:52.728905 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jun 30 18:22:52.728905 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: Jun 30 18:22:52.730104 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507fe3fa-1ba6-4124-84cd-6ca4753978cc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:52.750008 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 745/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 30 18:22:52 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:22:54.820150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610586d-4a22-4370-8c76-0da7fc2c4268 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:54.820675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610586d-4a22-4370-8c76-0da7fc2c4268 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:54.820675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610586d-4a22-4370-8c76-0da7fc2c4268 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:54.822630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 734/1479] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:22:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:22:54.827966 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71d2da-73da-49dd-aabc-b79c3e34fa5f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:54.828400 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71d2da-73da-49dd-aabc-b79c3e34fa5f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:54.828533 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71d2da-73da-49dd-aabc-b79c3e34fa5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:54.846712 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5b71d2da-73da-49dd-aabc-b79c3e34fa5f None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:54.847914 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5b71d2da-73da-49dd-aabc-b79c3e34fa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:54.855907 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 746/1480] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:22:54.899324 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610586d-4a22-4370-8c76-0da7fc2c4268 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:54.914608 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610586d-4a22-4370-8c76-0da7fc2c4268 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:54.920592 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46ff49-b7ea-4277-a8d3-80381d25f99d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:54.921221 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46ff49-b7ea-4277-a8d3-80381d25f99d service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:54.921503 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46ff49-b7ea-4277-a8d3-80381d25f99d service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:54.921811 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46ff49-b7ea-4277-a8d3-80381d25f99d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:54.930349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46ff49-b7ea-4277-a8d3-80381d25f99d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:54.932634 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46ff49-b7ea-4277-a8d3-80381d25f99d service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:54.943313 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 735/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:22:57.190444 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737cfa8-dca4-4257-9cb9-d9844cd56bb4 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:57.191016 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737cfa8-dca4-4257-9cb9-d9844cd56bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:57.191016 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737cfa8-dca4-4257-9cb9-d9844cd56bb4 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:57.192390 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 747/1482] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:22:57.198128 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f3674-e6fd-4991-adee-f63fe8803ae6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:57.198592 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f3674-e6fd-4991-adee-f63fe8803ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:57.198749 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f3674-e6fd-4991-adee-f63fe8803ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:57.216827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-449f3674-e6fd-4991-adee-f63fe8803ae6 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:57.218064 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-449f3674-e6fd-4991-adee-f63fe8803ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:57.226301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 736/1483] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:22:57.265065 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737cfa8-dca4-4257-9cb9-d9844cd56bb4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:57.276675 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737cfa8-dca4-4257-9cb9-d9844cd56bb4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:57.280333 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d4a4f-face-4c0a-b787-fcbc3b200b2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:57.280914 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d4a4f-face-4c0a-b787-fcbc3b200b2f service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:57.281206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d4a4f-face-4c0a-b787-fcbc3b200b2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:57.281491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d4a4f-face-4c0a-b787-fcbc3b200b2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:57.291125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d4a4f-face-4c0a-b787-fcbc3b200b2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:57.292729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d4a4f-face-4c0a-b787-fcbc3b200b2f service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:57.302196 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 748/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:22:59.448913 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935eb7c8-cc92-4586-9cc0-d15690e588e7 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:59.449485 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935eb7c8-cc92-4586-9cc0-d15690e588e7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:59.449485 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935eb7c8-cc92-4586-9cc0-d15690e588e7 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:59.450897 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 737/1485] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:22:59.456580 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b7f36-c7f0-4dda-b034-87fb4ce61f00 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:59.457143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b7f36-c7f0-4dda-b034-87fb4ce61f00 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:59.457143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b7f36-c7f0-4dda-b034-87fb4ce61f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:59.475999 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2e9b7f36-c7f0-4dda-b034-87fb4ce61f00 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:59.477234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2e9b7f36-c7f0-4dda-b034-87fb4ce61f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:59.486209 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 749/1486] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:22:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:22:59.525137 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935eb7c8-cc92-4586-9cc0-d15690e588e7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:59.535882 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935eb7c8-cc92-4586-9cc0-d15690e588e7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:59.539386 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9119e0-dc33-4f25-81e9-a37cacc7fc82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:59.540009 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9119e0-dc33-4f25-81e9-a37cacc7fc82 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:59.540321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9119e0-dc33-4f25-81e9-a37cacc7fc82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:59.540597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9119e0-dc33-4f25-81e9-a37cacc7fc82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:59.549883 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9119e0-dc33-4f25-81e9-a37cacc7fc82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:59.551325 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9119e0-dc33-4f25-81e9-a37cacc7fc82 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:59.562195 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 738/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:23:01.801734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204fbe11-df5a-4f49-a441-baf4a04d6563 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:01.802381 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204fbe11-df5a-4f49-a441-baf4a04d6563 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:01.802568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204fbe11-df5a-4f49-a441-baf4a04d6563 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:01.803967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 750/1488] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:23:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:23:01.809690 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd8c72-acaa-4924-8312-edc86b9d78e7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:01.810115 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd8c72-acaa-4924-8312-edc86b9d78e7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:01.810264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd8c72-acaa-4924-8312-edc86b9d78e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:01.828465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-35dd8c72-acaa-4924-8312-edc86b9d78e7 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:23:01.829703 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-35dd8c72-acaa-4924-8312-edc86b9d78e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:23:01.838075 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 739/1489] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:23:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:23:01.878672 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204fbe11-df5a-4f49-a441-baf4a04d6563 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:01.889365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204fbe11-df5a-4f49-a441-baf4a04d6563 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:01.893020 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f058a7b-e4f6-48f1-8877-b0e0618f5d2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:01.893686 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f058a7b-e4f6-48f1-8877-b0e0618f5d2d service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:01.893971 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f058a7b-e4f6-48f1-8877-b0e0618f5d2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:01.894248 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f058a7b-e4f6-48f1-8877-b0e0618f5d2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:01.903847 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f058a7b-e4f6-48f1-8877-b0e0618f5d2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:01.905270 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f058a7b-e4f6-48f1-8877-b0e0618f5d2d service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:01.914634 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 751/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:23:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:23:04.234963 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cad595-a4fb-4839-8f57-48ea7e96775d None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:04.235607 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cad595-a4fb-4839-8f57-48ea7e96775d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:04.235607 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cad595-a4fb-4839-8f57-48ea7e96775d None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:04.236929 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 740/1491] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 30 18:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:23:04.242309 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3cad6-1a1d-4d92-b89d-8c55a4482d0e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:04.242721 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3cad6-1a1d-4d92-b89d-8c55a4482d0e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:04.243022 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3cad6-1a1d-4d92-b89d-8c55a4482d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:04.261967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-eeb3cad6-1a1d-4d92-b89d-8c55a4482d0e None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:23:04.263263 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-eeb3cad6-1a1d-4d92-b89d-8c55a4482d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:23:04.271521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 752/1492] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 30 18:23:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:23:04.312066 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cad595-a4fb-4839-8f57-48ea7e96775d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:04.322972 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cad595-a4fb-4839-8f57-48ea7e96775d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:04.326518 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aeebeb-3e7a-422c-a0e3-d06c9c18cbce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:04.327151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aeebeb-3e7a-422c-a0e3-d06c9c18cbce service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:04.327437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aeebeb-3e7a-422c-a0e3-d06c9c18cbce service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:04.327716 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aeebeb-3e7a-422c-a0e3-d06c9c18cbce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:04.337169 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aeebeb-3e7a-422c-a0e3-d06c9c18cbce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:04.338645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aeebeb-3e7a-422c-a0e3-d06c9c18cbce service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:04.348219 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 741/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:23:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:23:14.255485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=33 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:23:14.343928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=33 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:24:43.166685 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab066389-f314-4f45-b3fc-3725a75c6871 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:24:43.167271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab066389-f314-4f45-b3fc-3725a75c6871 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:24:43.167271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab066389-f314-4f45-b3fc-3725a75c6871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:24:43.185739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ab066389-f314-4f45-b3fc-3725a75c6871 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:24:43.186946 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ab066389-f314-4f45-b3fc-3725a75c6871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:24:43.197060 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 753/1494] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:24:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:24:43.204468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0f1f4-7c2a-4bb3-8815-02b452452be4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:24:43.204950 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0f1f4-7c2a-4bb3-8815-02b452452be4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:24:43.205043 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0f1f4-7c2a-4bb3-8815-02b452452be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:24:43.222314 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5fb0f1f4-7c2a-4bb3-8815-02b452452be4 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:24:43.223639 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5fb0f1f4-7c2a-4bb3-8815-02b452452be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:24:43.231332 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 742/1495] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:24:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:24:43.235844 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab066389-f314-4f45-b3fc-3725a75c6871 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:24:43.246170 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab066389-f314-4f45-b3fc-3725a75c6871 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:24:43.251040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed42f3c-60bf-4869-a741-c7abbade30aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:24:43.251698 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed42f3c-60bf-4869-a741-c7abbade30aa admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:24:43.251988 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed42f3c-60bf-4869-a741-c7abbade30aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:24:43.252267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed42f3c-60bf-4869-a741-c7abbade30aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:24:43.253050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed42f3c-60bf-4869-a741-c7abbade30aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:24:43.254096 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed42f3c-60bf-4869-a741-c7abbade30aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:24:43.255116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-8ed42f3c-60bf-4869-a741-c7abbade30aa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 18:24:43.255116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jun 30 18:24:43.255116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: Jun 30 18:24:43.256014 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed42f3c-60bf-4869-a741-c7abbade30aa admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:24:43.267489 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 754/1496] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:24:43.275676 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb0f1f4-7c2a-4bb3-8815-02b452452be4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:24:43.285116 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb0f1f4-7c2a-4bb3-8815-02b452452be4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:24:43.289728 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5402a3-2f6e-428c-99a6-412587d50983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:24:43.290378 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5402a3-2f6e-428c-99a6-412587d50983 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:24:43.290667 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5402a3-2f6e-428c-99a6-412587d50983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:24:43.290950 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5402a3-2f6e-428c-99a6-412587d50983 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:24:43.291641 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5402a3-2f6e-428c-99a6-412587d50983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:24:43.293403 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-8d5402a3-2f6e-428c-99a6-412587d50983 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 18:24:43.293403 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jun 30 18:24:43.293403 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: Jun 30 18:24:43.294632 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5402a3-2f6e-428c-99a6-412587d50983 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:24:43.330318 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 743/1497] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:24:43 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:24:43.335290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed42f3c-60bf-4869-a741-c7abbade30aa admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:24:43.345024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed42f3c-60bf-4869-a741-c7abbade30aa admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:24:43.400925 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc267b-2085-4a77-8598-f00ca49e363e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:24:43.401613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc267b-2085-4a77-8598-f00ca49e363e admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:24:43.401903 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc267b-2085-4a77-8598-f00ca49e363e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:24:43.402188 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc267b-2085-4a77-8598-f00ca49e363e admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:24:43.402883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc267b-2085-4a77-8598-f00ca49e363e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:24:43.403900 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-63fc267b-2085-4a77-8598-f00ca49e363e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 18:24:43.403900 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jun 30 18:24:43.403900 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: Jun 30 18:24:43.405114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc267b-2085-4a77-8598-f00ca49e363e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:24:43.426201 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 755/1498] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:24:43 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:24:43.430385 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5402a3-2f6e-428c-99a6-412587d50983 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:24:43.440703 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5402a3-2f6e-428c-99a6-412587d50983 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:24:43.445440 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33877d6-c0a5-4ab8-b379-2c887f659d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:24:43.446120 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33877d6-c0a5-4ab8-b379-2c887f659d77 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:24:43.446523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33877d6-c0a5-4ab8-b379-2c887f659d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:24:43.446807 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33877d6-c0a5-4ab8-b379-2c887f659d77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:24:43.447527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33877d6-c0a5-4ab8-b379-2c887f659d77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:24:43.448526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-a33877d6-c0a5-4ab8-b379-2c887f659d77 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 18:24:43.448526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jun 30 18:24:43.448526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: Jun 30 18:24:43.449402 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33877d6-c0a5-4ab8-b379-2c887f659d77 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:24:43.462261 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 744/1499] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:24:43.467721 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc267b-2085-4a77-8598-f00ca49e363e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:24:43.480198 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc267b-2085-4a77-8598-f00ca49e363e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:24:43.484935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408895fb-0dc4-4e1e-ae02-ae63e14b0580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:24:43.485607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408895fb-0dc4-4e1e-ae02-ae63e14b0580 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:24:43.485906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408895fb-0dc4-4e1e-ae02-ae63e14b0580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:24:43.486192 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408895fb-0dc4-4e1e-ae02-ae63e14b0580 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:24:43.486918 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408895fb-0dc4-4e1e-ae02-ae63e14b0580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:24:43.488806 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-408895fb-0dc4-4e1e-ae02-ae63e14b0580 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 18:24:43.488806 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jun 30 18:24:43.488806 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: Jun 30 18:24:43.489726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408895fb-0dc4-4e1e-ae02-ae63e14b0580 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:24:43.502895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 756/1500] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:24:43.507068 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33877d6-c0a5-4ab8-b379-2c887f659d77 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:24:43.516832 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33877d6-c0a5-4ab8-b379-2c887f659d77 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:24:43.521292 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa83ee6-4591-4e65-aa25-c4bb0bc08e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:24:43.522192 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa83ee6-4591-4e65-aa25-c4bb0bc08e7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:24:43.522476 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa83ee6-4591-4e65-aa25-c4bb0bc08e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:24:43.522751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa83ee6-4591-4e65-aa25-c4bb0bc08e7c admin admin] PATH_INFO: `/v3/projects/665cafbc4a034126a411156199e18c8d/users/ee2f1167e4fe4760aff8254a99871ae3/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:24:43.537695 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa83ee6-4591-4e65-aa25-c4bb0bc08e7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=665cafbc4a034126a411156199e18c8d, user_id=ee2f1167e4fe4760aff8254a99871ae3, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:24:43.538642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-caa83ee6-4591-4e65-aa25-c4bb0bc08e7c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 18:24:43.538642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jun 30 18:24:43.538642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: Jun 30 18:24:43.540569 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa83ee6-4591-4e65-aa25-c4bb0bc08e7c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:24:43.554170 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 745/1501] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:24:43 2022] PUT /identity/v3/projects/665cafbc4a034126a411156199e18c8d/users/ee2f1167e4fe4760aff8254a99871ae3/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:24:43.566652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87649906-d9a9-4caa-9f56-0d99eaba6410 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:24:43.566949 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87649906-d9a9-4caa-9f56-0d99eaba6410 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:24:43.567237 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87649906-d9a9-4caa-9f56-0d99eaba6410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:24:43.592856 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-87649906-d9a9-4caa-9f56-0d99eaba6410 None None] MFA Rules not processed for user `ee2f1167e4fe4760aff8254a99871ae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:24:43.594120 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-87649906-d9a9-4caa-9f56-0d99eaba6410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:24:43.657545 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 757/1502] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:24:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:24:43.671519 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79ab90-1cc1-45ac-87e6-a062610f24c3 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:24:43.671805 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79ab90-1cc1-45ac-87e6-a062610f24c3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:24:43.672083 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79ab90-1cc1-45ac-87e6-a062610f24c3 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:24:43.673458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 746/1503] 10.222.0.33 () {54 vars in 963 bytes} [Thu Jun 30 18:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:24:43.678985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7710766-0fab-49f5-a5d0-f1930421affe None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:24:43.679308 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7710766-0fab-49f5-a5d0-f1930421affe None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:24:43.679594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7710766-0fab-49f5-a5d0-f1930421affe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:24:43.697696 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b7710766-0fab-49f5-a5d0-f1930421affe None None] MFA Rules not processed for user `92c70ce830ea4dd981716d622b16d595`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:24:43.698906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b7710766-0fab-49f5-a5d0-f1930421affe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:24:43.744373 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:24:43.752663 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 758/1504] 10.222.0.33 () {58 vars in 1060 bytes} [Thu Jun 30 18:24:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:24:43.760393 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79ab90-1cc1-45ac-87e6-a062610f24c3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:24:43.771435 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79ab90-1cc1-45ac-87e6-a062610f24c3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:24:43.774889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1d379-a98e-4520-9224-1ef28fe743a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:24:43.775525 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1d379-a98e-4520-9224-1ef28fe743a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:24:43.775806 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1d379-a98e-4520-9224-1ef28fe743a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:24:43.776083 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1d379-a98e-4520-9224-1ef28fe743a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:24:43.784115 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1d379-a98e-4520-9224-1ef28fe743a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:24:43.786279 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1d379-a98e-4520-9224-1ef28fe743a5 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:24:43.794876 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 747/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:24:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:24:43.836805 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1adf0cf-dc85-4cab-b3a7-830e07facc9f None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:24:43.837102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1adf0cf-dc85-4cab-b3a7-830e07facc9f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:24:43.837387 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1adf0cf-dc85-4cab-b3a7-830e07facc9f None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:24:43.838699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 759/1506] 10.222.0.33 () {52 vars in 829 bytes} [Thu Jun 30 18:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:24:43.842367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1d379-a98e-4520-9224-1ef28fe743a5 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:24:43.903804 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1d379-a98e-4520-9224-1ef28fe743a5 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:24:43.908299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309650cd-0c98-435b-9970-a6b1e64df1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:24:43.909155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309650cd-0c98-435b-9970-a6b1e64df1d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:24:43.909445 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309650cd-0c98-435b-9970-a6b1e64df1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:24:43.910344 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309650cd-0c98-435b-9970-a6b1e64df1d3 admin admin] PATH_INFO: `/v3/users/ee2f1167e4fe4760aff8254a99871ae3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:24:43.911547 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309650cd-0c98-435b-9970-a6b1e64df1d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee2f1167e4fe4760aff8254a99871ae3)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:24:43.912570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-309650cd-0c98-435b-9970-a6b1e64df1d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 18:24:43.912570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jun 30 18:24:43.912570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: Jun 30 18:24:43.913826 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309650cd-0c98-435b-9970-a6b1e64df1d3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:24:43.980389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-309650cd-0c98-435b-9970-a6b1e64df1d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee2f1167e4fe4760aff8254a99871ae3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:24:43.981023 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-309650cd-0c98-435b-9970-a6b1e64df1d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee2f1167e4fe4760aff8254a99871ae3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:24:43.999175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-309650cd-0c98-435b-9970-a6b1e64df1d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee2f1167e4fe4760aff8254a99871ae3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:24:44.008326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-309650cd-0c98-435b-9970-a6b1e64df1d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee2f1167e4fe4760aff8254a99871ae3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:24:44.009050 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-309650cd-0c98-435b-9970-a6b1e64df1d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee2f1167e4fe4760aff8254a99871ae3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:24:44.017617 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:24:44.030064 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 748/1507] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:24:43 2022] DELETE /identity/v3/users/ee2f1167e4fe4760aff8254a99871ae3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:24:44.035232 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d1c4e-2d20-44ef-abec-18cba25e9268 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:24:44.035546 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d1c4e-2d20-44ef-abec-18cba25e9268 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:24:44.035825 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d1c4e-2d20-44ef-abec-18cba25e9268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:24:44.052788 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2d3d1c4e-2d20-44ef-abec-18cba25e9268 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:24:44.054012 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2d3d1c4e-2d20-44ef-abec-18cba25e9268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:24:44.111870 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 760/1508] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:24:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:24:44.122365 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309650cd-0c98-435b-9970-a6b1e64df1d3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:24:44.123888 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-309650cd-0c98-435b-9970-a6b1e64df1d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:24:44.182926 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309650cd-0c98-435b-9970-a6b1e64df1d3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:24:44.186457 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba9650-f0f0-4178-b774-ad360a6ca379 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:24:44.187101 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba9650-f0f0-4178-b774-ad360a6ca379 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:24:44.187385 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba9650-f0f0-4178-b774-ad360a6ca379 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:24:44.187661 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba9650-f0f0-4178-b774-ad360a6ca379 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:24:44.196961 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba9650-f0f0-4178-b774-ad360a6ca379 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:24:44.198453 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba9650-f0f0-4178-b774-ad360a6ca379 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:24:44.208208 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 749/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:24:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:24:44.658469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d1c4e-2d20-44ef-abec-18cba25e9268 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:24:44.660520 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2d3d1c4e-2d20-44ef-abec-18cba25e9268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:24:44.672200 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d1c4e-2d20-44ef-abec-18cba25e9268 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:24:44.678936 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2d7f8-f0f9-4499-90fd-8bffdb8be37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:24:44.679797 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2d7f8-f0f9-4499-90fd-8bffdb8be37e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:24:44.680084 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2d7f8-f0f9-4499-90fd-8bffdb8be37e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:24:44.680364 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2d7f8-f0f9-4499-90fd-8bffdb8be37e admin admin] PATH_INFO: `/v3/projects/665cafbc4a034126a411156199e18c8d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:24:44.681563 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2d7f8-f0f9-4499-90fd-8bffdb8be37e admin admin] RBAC: Authorizing `identity:delete_project(project_id=665cafbc4a034126a411156199e18c8d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:24:44.683475 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-fce2d7f8-f0f9-4499-90fd-8bffdb8be37e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 18:24:44.683475 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jun 30 18:24:44.683475 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: Jun 30 18:24:44.684878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2d7f8-f0f9-4499-90fd-8bffdb8be37e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:24:44.749796 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fce2d7f8-f0f9-4499-90fd-8bffdb8be37e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '665cafbc4a034126a411156199e18c8d', 'request_id': 'req-fce2d7f8-f0f9-4499-90fd-8bffdb8be37e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:24:44.765053 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fce2d7f8-f0f9-4499-90fd-8bffdb8be37e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '665cafbc4a034126a411156199e18c8d', 'request_id': 'req-fce2d7f8-f0f9-4499-90fd-8bffdb8be37e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:24:44.771573 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fce2d7f8-f0f9-4499-90fd-8bffdb8be37e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '665cafbc4a034126a411156199e18c8d', 'request_id': 'req-fce2d7f8-f0f9-4499-90fd-8bffdb8be37e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:24:44.786165 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fce2d7f8-f0f9-4499-90fd-8bffdb8be37e admin admin] The token cache is being invalidate because project 665cafbc4a034126a411156199e18c8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:24:44.786459 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fce2d7f8-f0f9-4499-90fd-8bffdb8be37e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:24:44.787171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fce2d7f8-f0f9-4499-90fd-8bffdb8be37e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:24:44.788698 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 761/1510] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:24:44 2022] DELETE /identity/v3/projects/665cafbc4a034126a411156199e18c8d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:24:54.203555 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=29 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:24:54.782548 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:21.922612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549fa75-5a24-4baa-a121-873defa0c055 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:21.923232 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549fa75-5a24-4baa-a121-873defa0c055 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:21.923232 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549fa75-5a24-4baa-a121-873defa0c055 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:21.924552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 18:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:26:21.929350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a52132-ae37-416f-a7cd-cd1cdd561dca None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:21.929829 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a52132-ae37-416f-a7cd-cd1cdd561dca None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:21.929939 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a52132-ae37-416f-a7cd-cd1cdd561dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:21.949647 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-14a52132-ae37-416f-a7cd-cd1cdd561dca None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:21.950867 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-14a52132-ae37-416f-a7cd-cd1cdd561dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:22.007924 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 18:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:22.162257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549fa75-5a24-4baa-a121-873defa0c055 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:22.163972 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2549fa75-5a24-4baa-a121-873defa0c055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:22.238720 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549fa75-5a24-4baa-a121-873defa0c055 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:22.244665 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55085e-3c7f-49db-b557-b306e046e5f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:22.245304 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55085e-3c7f-49db-b557-b306e046e5f9 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:22.245602 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55085e-3c7f-49db-b557-b306e046e5f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:22.245904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55085e-3c7f-49db-b557-b306e046e5f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:22.253984 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55085e-3c7f-49db-b557-b306e046e5f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:22.255586 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55085e-3c7f-49db-b557-b306e046e5f9 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:22.264880 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 751/1513] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:23.102472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbf282-9e7a-407b-ab67-471c47127aef None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:23.102859 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbf282-9e7a-407b-ab67-471c47127aef None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:23.103701 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbf282-9e7a-407b-ab67-471c47127aef None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:23.104898 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 18:26:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:26:23.109787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb095e6-7059-4745-97a5-a9e70c5425d2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:23.110190 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb095e6-7059-4745-97a5-a9e70c5425d2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:23.110364 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb095e6-7059-4745-97a5-a9e70c5425d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:23.128753 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8eb095e6-7059-4745-97a5-a9e70c5425d2 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:23.129970 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8eb095e6-7059-4745-97a5-a9e70c5425d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:23.138174 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 18:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:23.292284 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbf282-9e7a-407b-ab67-471c47127aef None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:23.306464 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbf282-9e7a-407b-ab67-471c47127aef None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:23.312148 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21385cd3-740d-4ba5-b5c6-061e2cb61cf7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:23.312787 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21385cd3-740d-4ba5-b5c6-061e2cb61cf7 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:23.313077 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21385cd3-740d-4ba5-b5c6-061e2cb61cf7 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:23.313363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21385cd3-740d-4ba5-b5c6-061e2cb61cf7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:23.321465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21385cd3-740d-4ba5-b5c6-061e2cb61cf7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:23.323334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21385cd3-740d-4ba5-b5c6-061e2cb61cf7 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:23.332757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 764/1516] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:24.182079 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5a600-6e4a-4a35-9654-3feaa2d6bc44 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:24.182916 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5a600-6e4a-4a35-9654-3feaa2d6bc44 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:24.182916 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5a600-6e4a-4a35-9654-3feaa2d6bc44 None None] PATH_INFO: `/v3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:24.184223 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 753/1517] 10.222.0.18 () {54 vars in 925 bytes} [Thu Jun 30 18:26:24 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:26:24.190409 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5812a-2b56-4128-affb-d312a8d8aed9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:24.190815 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5812a-2b56-4128-affb-d312a8d8aed9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:24.190982 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5812a-2b56-4128-affb-d312a8d8aed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:24.208875 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a5c5812a-2b56-4128-affb-d312a8d8aed9 None None] MFA Rules not processed for user `49325c1733174e0c98b3475a10c9b5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:24.210108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a5c5812a-2b56-4128-affb-d312a8d8aed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:24.271801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 765/1518] 10.222.0.18 () {58 vars in 1015 bytes} [Thu Jun 30 18:26:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:26:33.123647 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=37 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:33.566286 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c854f-372f-4735-aa17-711b50b175e8 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:33.566507 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c854f-372f-4735-aa17-711b50b175e8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:33.566770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c854f-372f-4735-aa17-711b50b175e8 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:33.568154 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 754/1519] 10.222.0.33 () {54 vars in 960 bytes} [Thu Jun 30 18:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:26:33.573791 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960994f9-bab6-4e6a-8e0f-8a143d694d3e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:33.574293 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960994f9-bab6-4e6a-8e0f-8a143d694d3e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:33.574400 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960994f9-bab6-4e6a-8e0f-8a143d694d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:33.593247 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-960994f9-bab6-4e6a-8e0f-8a143d694d3e None None] MFA Rules not processed for user `49325c1733174e0c98b3475a10c9b5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:33.594496 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-960994f9-bab6-4e6a-8e0f-8a143d694d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:33.603057 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 766/1520] 10.222.0.33 () {58 vars in 1057 bytes} [Thu Jun 30 18:26:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:26:33.610537 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c854f-372f-4735-aa17-711b50b175e8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:33.623846 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c854f-372f-4735-aa17-711b50b175e8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:33.628674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6fb16-8cf8-4b5f-a6db-8c1313618c28 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49325c1733174e0c98b3475a10c9b5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:33.629320 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6fb16-8cf8-4b5f-a6db-8c1313618c28 service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:33.629626 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6fb16-8cf8-4b5f-a6db-8c1313618c28 service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:33.629908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6fb16-8cf8-4b5f-a6db-8c1313618c28 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:33.638628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6fb16-8cf8-4b5f-a6db-8c1313618c28 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:33.640223 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6fb16-8cf8-4b5f-a6db-8c1313618c28 service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:33.651572 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 755/1521] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:34.589679 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6d294-da49-4b00-9fa8-8e0970fec15e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:34.590209 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6d294-da49-4b00-9fa8-8e0970fec15e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:34.590307 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6d294-da49-4b00-9fa8-8e0970fec15e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:34.591713 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 767/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 18:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:26:34.596568 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5703371-d607-4d08-a9de-f6b43479aacf None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:34.596941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5703371-d607-4d08-a9de-f6b43479aacf None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:34.597141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5703371-d607-4d08-a9de-f6b43479aacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:34.614889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a5703371-d607-4d08-a9de-f6b43479aacf None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:34.616104 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a5703371-d607-4d08-a9de-f6b43479aacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:34.624652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 756/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 18:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:34.779998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6d294-da49-4b00-9fa8-8e0970fec15e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:34.795518 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6d294-da49-4b00-9fa8-8e0970fec15e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:34.800908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65f5f1-760b-4a0f-a841-bc832bc284f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:34.801541 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65f5f1-760b-4a0f-a841-bc832bc284f8 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:34.801858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65f5f1-760b-4a0f-a841-bc832bc284f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:34.802141 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65f5f1-760b-4a0f-a841-bc832bc284f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:34.810869 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65f5f1-760b-4a0f-a841-bc832bc284f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:34.812510 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65f5f1-760b-4a0f-a841-bc832bc284f8 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:34.822169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 768/1524] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:35.667639 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd7039-a89d-4cf6-8a7f-99c47fc74579 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:35.668181 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd7039-a89d-4cf6-8a7f-99c47fc74579 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:35.668181 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd7039-a89d-4cf6-8a7f-99c47fc74579 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:35.669598 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 757/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 18:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:26:35.674454 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e2cd2-f46d-43fb-95ab-6a57ce51a349 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:35.674861 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e2cd2-f46d-43fb-95ab-6a57ce51a349 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:35.675018 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e2cd2-f46d-43fb-95ab-6a57ce51a349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:35.692832 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-653e2cd2-f46d-43fb-95ab-6a57ce51a349 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:35.694063 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-653e2cd2-f46d-43fb-95ab-6a57ce51a349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:35.702512 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 769/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 18:26:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:35.854990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd7039-a89d-4cf6-8a7f-99c47fc74579 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:35.870401 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd7039-a89d-4cf6-8a7f-99c47fc74579 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:35.875876 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57d1f8-aea9-4de5-b481-17519e13e2a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:35.876506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57d1f8-aea9-4de5-b481-17519e13e2a1 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:35.876795 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57d1f8-aea9-4de5-b481-17519e13e2a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:35.877076 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57d1f8-aea9-4de5-b481-17519e13e2a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:35.885245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c57d1f8-aea9-4de5-b481-17519e13e2a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:35.886874 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c57d1f8-aea9-4de5-b481-17519e13e2a1 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:35.896260 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 758/1527] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:39.986371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b25a13-6fe6-42b2-aa53-4b79bb09dd5e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:39.986924 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b25a13-6fe6-42b2-aa53-4b79bb09dd5e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:39.986924 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b25a13-6fe6-42b2-aa53-4b79bb09dd5e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:39.988362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 770/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 18:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:26:39.994376 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176dd34-08b3-4d41-b269-031d658688f2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:39.994796 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176dd34-08b3-4d41-b269-031d658688f2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:39.994936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176dd34-08b3-4d41-b269-031d658688f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:40.013038 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e176dd34-08b3-4d41-b269-031d658688f2 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:40.014295 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e176dd34-08b3-4d41-b269-031d658688f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:40.024987 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 759/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 18:26:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:40.059210 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b25a13-6fe6-42b2-aa53-4b79bb09dd5e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:40.060947 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-72b25a13-6fe6-42b2-aa53-4b79bb09dd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:40.118002 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b25a13-6fe6-42b2-aa53-4b79bb09dd5e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:40.122330 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e284442-a8a6-4c87-bf15-70831f1b09fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:40.122977 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e284442-a8a6-4c87-bf15-70831f1b09fe service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:40.123267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e284442-a8a6-4c87-bf15-70831f1b09fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:40.123552 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e284442-a8a6-4c87-bf15-70831f1b09fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:40.133217 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e284442-a8a6-4c87-bf15-70831f1b09fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:40.135455 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e284442-a8a6-4c87-bf15-70831f1b09fe service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:40.140689 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:40.146069 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 771/1530] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:42.222382 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cc9ea-a202-488c-8274-0628e10643e7 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:42.222938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cc9ea-a202-488c-8274-0628e10643e7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:42.222938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cc9ea-a202-488c-8274-0628e10643e7 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:42.224334 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 760/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 18:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:26:42.230434 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3508fd8-12d2-4115-afca-f6299df142d1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:42.230841 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3508fd8-12d2-4115-afca-f6299df142d1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:42.231005 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3508fd8-12d2-4115-afca-f6299df142d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:42.249264 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a3508fd8-12d2-4115-afca-f6299df142d1 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:42.250496 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a3508fd8-12d2-4115-afca-f6299df142d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:42.258207 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 772/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 18:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:26:42.295175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0cc9ea-a202-488c-8274-0628e10643e7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:42.305852 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0cc9ea-a202-488c-8274-0628e10643e7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:42.309506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cff24d-fdcb-4b4f-9d12-ff75cba4c8f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:42.310157 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cff24d-fdcb-4b4f-9d12-ff75cba4c8f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:42.310439 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cff24d-fdcb-4b4f-9d12-ff75cba4c8f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:42.310714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cff24d-fdcb-4b4f-9d12-ff75cba4c8f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:42.319362 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cff24d-fdcb-4b4f-9d12-ff75cba4c8f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:42.320777 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cff24d-fdcb-4b4f-9d12-ff75cba4c8f7 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:42.330012 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 761/1533] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:45.837027 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b5f64-de92-4627-947e-d1191d48c20e None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:45.838222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b5f64-de92-4627-947e-d1191d48c20e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:45.838222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b5f64-de92-4627-947e-d1191d48c20e None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:45.839666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 773/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 18:26:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:26:45.844842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8283b-a048-4951-b854-627ecaef88c2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:45.845273 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8283b-a048-4951-b854-627ecaef88c2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:45.845387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8283b-a048-4951-b854-627ecaef88c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:45.864276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-dfd8283b-a048-4951-b854-627ecaef88c2 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:45.865472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dfd8283b-a048-4951-b854-627ecaef88c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:45.873355 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 762/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 18:26:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:46.191970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b5f64-de92-4627-947e-d1191d48c20e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:46.194602 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-815b5f64-de92-4627-947e-d1191d48c20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:46.251740 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b5f64-de92-4627-947e-d1191d48c20e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:46.255330 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f4ce1-0924-4ab2-befb-48530be45b8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:46.255967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f4ce1-0924-4ab2-befb-48530be45b8b service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:46.256280 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f4ce1-0924-4ab2-befb-48530be45b8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:46.256567 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f4ce1-0924-4ab2-befb-48530be45b8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:46.265984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f4ce1-0924-4ab2-befb-48530be45b8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:46.267419 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f4ce1-0924-4ab2-befb-48530be45b8b service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:46.277430 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 774/1536] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:48.332732 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7590c9-4a64-47a9-a4d2-6e38e50124e0 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:48.333296 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7590c9-4a64-47a9-a4d2-6e38e50124e0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:48.333296 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7590c9-4a64-47a9-a4d2-6e38e50124e0 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:48.334604 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 763/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 18:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:26:48.340299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ee265-9e7e-4e46-85d3-a573d60d398b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:48.340754 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ee265-9e7e-4e46-85d3-a573d60d398b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:48.340862 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ee265-9e7e-4e46-85d3-a573d60d398b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:48.359607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c94ee265-9e7e-4e46-85d3-a573d60d398b None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:48.360829 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c94ee265-9e7e-4e46-85d3-a573d60d398b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:48.368947 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 775/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 18:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:48.668698 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7590c9-4a64-47a9-a4d2-6e38e50124e0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:48.670416 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1e7590c9-4a64-47a9-a4d2-6e38e50124e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:48.684687 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7590c9-4a64-47a9-a4d2-6e38e50124e0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:48.688156 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33109600-743c-4e0b-a89e-4df2c5f8eab6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:48.688783 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33109600-743c-4e0b-a89e-4df2c5f8eab6 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:48.689067 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33109600-743c-4e0b-a89e-4df2c5f8eab6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:48.689349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33109600-743c-4e0b-a89e-4df2c5f8eab6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:48.698815 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33109600-743c-4e0b-a89e-4df2c5f8eab6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:48.700245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33109600-743c-4e0b-a89e-4df2c5f8eab6 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:48.709630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 764/1539] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:58.354188 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=32 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:58.705424 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=33 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:19.521754 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d9e12-297c-48bf-8457-21379f1332b8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:19.522989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d9e12-297c-48bf-8457-21379f1332b8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:19.523512 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d9e12-297c-48bf-8457-21379f1332b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:19.550719 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-338d9e12-297c-48bf-8457-21379f1332b8 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:19.551979 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-338d9e12-297c-48bf-8457-21379f1332b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:19.561308 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 776/1540] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:28:19.572690 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c00c09-c31f-427f-9537-ee100c0b2f9b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:19.573297 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c00c09-c31f-427f-9537-ee100c0b2f9b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:19.573297 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c00c09-c31f-427f-9537-ee100c0b2f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:19.601823 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c8c00c09-c31f-427f-9537-ee100c0b2f9b None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:19.603049 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c8c00c09-c31f-427f-9537-ee100c0b2f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:19.616745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 765/1541] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:19.623081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d9e12-297c-48bf-8457-21379f1332b8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:19.639231 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d9e12-297c-48bf-8457-21379f1332b8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:19.643943 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a1ca7-5ae5-416b-bd0c-f83935894adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:19.644609 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a1ca7-5ae5-416b-bd0c-f83935894adc admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:19.644898 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a1ca7-5ae5-416b-bd0c-f83935894adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:19.645177 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a1ca7-5ae5-416b-bd0c-f83935894adc admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:19.645937 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a1ca7-5ae5-416b-bd0c-f83935894adc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:28:19.646256 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a1ca7-5ae5-416b-bd0c-f83935894adc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:19.647870 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a1ca7-5ae5-416b-bd0c-f83935894adc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:19.673010 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 777/1542] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:28:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:28:19.681804 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c00c09-c31f-427f-9537-ee100c0b2f9b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:19.700169 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c00c09-c31f-427f-9537-ee100c0b2f9b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:19.709245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51742a7f-08f8-416f-a701-96d278734283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:19.709967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51742a7f-08f8-416f-a701-96d278734283 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:19.710254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51742a7f-08f8-416f-a701-96d278734283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:19.710540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51742a7f-08f8-416f-a701-96d278734283 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:19.711257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51742a7f-08f8-416f-a701-96d278734283 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:19.714618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51742a7f-08f8-416f-a701-96d278734283 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:19.776940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 766/1543] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:28:19 2022] POST /identity/v3/projects => generated 391 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:19.781415 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a1ca7-5ae5-416b-bd0c-f83935894adc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:19.800218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a1ca7-5ae5-416b-bd0c-f83935894adc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:19.819816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b1f201-23ff-4dea-be50-26db183e2229 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:19.820131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b1f201-23ff-4dea-be50-26db183e2229 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:19.820387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b1f201-23ff-4dea-be50-26db183e2229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:19.852949 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-99b1f201-23ff-4dea-be50-26db183e2229 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:19.854857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-99b1f201-23ff-4dea-be50-26db183e2229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:19.884315 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7dae4e-f7d1-4c7c-a83f-5ed2e66bdb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:19.885522 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7dae4e-f7d1-4c7c-a83f-5ed2e66bdb4f admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:19.885836 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7dae4e-f7d1-4c7c-a83f-5ed2e66bdb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:19.886118 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7dae4e-f7d1-4c7c-a83f-5ed2e66bdb4f admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:19.886825 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7dae4e-f7d1-4c7c-a83f-5ed2e66bdb4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:19.888775 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7dae4e-f7d1-4c7c-a83f-5ed2e66bdb4f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:19.941501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 778/1544] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:28:19 2022] POST /identity/v3/users => generated 296 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:19.942897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029411f-0e15-47e9-97b4-ca6c50e90852 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:19.943204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029411f-0e15-47e9-97b4-ca6c50e90852 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:19.943494 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029411f-0e15-47e9-97b4-ca6c50e90852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:19.973631 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 767/1545] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:28:19.974113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a029411f-0e15-47e9-97b4-ca6c50e90852 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:19.974883 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a579b-a4ec-40ec-9c2b-a2cd4278c77a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:19.975179 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a579b-a4ec-40ec-9c2b-a2cd4278c77a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:19.975464 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a579b-a4ec-40ec-9c2b-a2cd4278c77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:19.976287 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a029411f-0e15-47e9-97b4-ca6c50e90852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:19.996388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 779/1546] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:19.998236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029411f-0e15-47e9-97b4-ca6c50e90852 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:20.016404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029411f-0e15-47e9-97b4-ca6c50e90852 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:20.018401 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7f7a579b-a4ec-40ec-9c2b-a2cd4278c77a None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:20.020199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7f7a579b-a4ec-40ec-9c2b-a2cd4278c77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:20.030395 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb321417-cc81-4035-82c9-54fc8d5652cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:20.031087 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb321417-cc81-4035-82c9-54fc8d5652cb admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.031376 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb321417-cc81-4035-82c9-54fc8d5652cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.031658 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb321417-cc81-4035-82c9-54fc8d5652cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.032395 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb321417-cc81-4035-82c9-54fc8d5652cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:20.034521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb321417-cc81-4035-82c9-54fc8d5652cb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:20.036544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 768/1547] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:20.038087 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150bb74-82b3-4ced-b89f-f0ba7d5a9682 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.038622 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150bb74-82b3-4ced-b89f-f0ba7d5a9682 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.039202 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150bb74-82b3-4ced-b89f-f0ba7d5a9682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.050472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 780/1548] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:20.050472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ca928-8204-4998-b0e0-8ce0e396b7c5 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.050472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ca928-8204-4998-b0e0-8ce0e396b7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.050472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ca928-8204-4998-b0e0-8ce0e396b7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.067509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d150bb74-82b3-4ced-b89f-f0ba7d5a9682 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:20.069172 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d150bb74-82b3-4ced-b89f-f0ba7d5a9682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:20.071588 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9c9ca928-8204-4998-b0e0-8ce0e396b7c5 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:20.073277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9c9ca928-8204-4998-b0e0-8ce0e396b7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:20.079384 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 769/1549] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:20.081785 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 781/1550] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:20.082899 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73992e-049c-4b22-addc-4e62876c34a6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.083188 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73992e-049c-4b22-addc-4e62876c34a6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.083540 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73992e-049c-4b22-addc-4e62876c34a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.084966 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76188f0f-edcf-463d-9b42-e9d1df554fbb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.085381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76188f0f-edcf-463d-9b42-e9d1df554fbb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.092067 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76188f0f-edcf-463d-9b42-e9d1df554fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.109005 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6b73992e-049c-4b22-addc-4e62876c34a6 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:20.110265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6b73992e-049c-4b22-addc-4e62876c34a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:20.111983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-76188f0f-edcf-463d-9b42-e9d1df554fbb None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:20.113356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-76188f0f-edcf-463d-9b42-e9d1df554fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:20.125087 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 782/1551] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:20.126677 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6bd23-8ef3-49e1-92ef-2badd36ada4f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.126969 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6bd23-8ef3-49e1-92ef-2badd36ada4f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.127252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6bd23-8ef3-49e1-92ef-2badd36ada4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.131348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 770/1552] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:20.132244 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76188f0f-edcf-463d-9b42-e9d1df554fbb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:20.142158 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76188f0f-edcf-463d-9b42-e9d1df554fbb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:20.146166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-08d6bd23-8ef3-49e1-92ef-2badd36ada4f None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:20.146736 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da56bd-aaf1-4581-be43-f8e58e79d5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:20.147368 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-08d6bd23-8ef3-49e1-92ef-2badd36ada4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:20.147507 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da56bd-aaf1-4581-be43-f8e58e79d5c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.147671 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da56bd-aaf1-4581-be43-f8e58e79d5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.147940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da56bd-aaf1-4581-be43-f8e58e79d5c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.148662 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da56bd-aaf1-4581-be43-f8e58e79d5c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:20.150997 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da56bd-aaf1-4581-be43-f8e58e79d5c6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:20.155878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 783/1553] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:20.157104 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdb302-5c4d-4501-8b53-ffb2eec40b90 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.157395 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdb302-5c4d-4501-8b53-ffb2eec40b90 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.157768 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdb302-5c4d-4501-8b53-ffb2eec40b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.164439 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 771/1554] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:20.165245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da56bd-aaf1-4581-be43-f8e58e79d5c6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:20.175230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da56bd-aaf1-4581-be43-f8e58e79d5c6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:20.175851 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c2bdb302-5c4d-4501-8b53-ffb2eec40b90 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:20.177046 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c2bdb302-5c4d-4501-8b53-ffb2eec40b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:20.179750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d09b6-c7ff-4edd-bced-21a062f2fb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:20.180389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d09b6-c7ff-4edd-bced-21a062f2fb76 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.180673 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d09b6-c7ff-4edd-bced-21a062f2fb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.180954 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d09b6-c7ff-4edd-bced-21a062f2fb76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.182290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d09b6-c7ff-4edd-bced-21a062f2fb76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:28:20.182602 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d09b6-c7ff-4edd-bced-21a062f2fb76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:20.183614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-4d9d09b6-c7ff-4edd-bced-21a062f2fb76 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 18:28:20.183614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jun 30 18:28:20.183614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: Jun 30 18:28:20.184505 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d09b6-c7ff-4edd-bced-21a062f2fb76 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:20.185719 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 784/1555] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:20.186925 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cdc13-8dd0-4113-b455-d1c2450f189d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.187578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cdc13-8dd0-4113-b455-d1c2450f189d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.187578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cdc13-8dd0-4113-b455-d1c2450f189d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.198313 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 772/1556] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:28:20.199659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9cb38-d1f6-48eb-a8c0-7e5e6d460cdb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.200286 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9cb38-d1f6-48eb-a8c0-7e5e6d460cdb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.200286 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9cb38-d1f6-48eb-a8c0-7e5e6d460cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.210038 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e84cdc13-8dd0-4113-b455-d1c2450f189d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:20.211265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e84cdc13-8dd0-4113-b455-d1c2450f189d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:20.218621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-68f9cb38-d1f6-48eb-a8c0-7e5e6d460cdb None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:20.219442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 785/1557] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:20.219801 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-68f9cb38-d1f6-48eb-a8c0-7e5e6d460cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:20.220796 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d96ae-b1cd-434d-a233-c3146267c33d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.221093 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d96ae-b1cd-434d-a233-c3146267c33d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.221379 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d96ae-b1cd-434d-a233-c3146267c33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.227764 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 773/1558] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:20.228616 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9cb38-d1f6-48eb-a8c0-7e5e6d460cdb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:20.238628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9cb38-d1f6-48eb-a8c0-7e5e6d460cdb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:20.239825 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-168d96ae-b1cd-434d-a233-c3146267c33d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:20.241037 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-168d96ae-b1cd-434d-a233-c3146267c33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:20.243341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397fea8-4ede-4d38-af63-7492fa11709a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:20.244658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397fea8-4ede-4d38-af63-7492fa11709a admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.244950 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397fea8-4ede-4d38-af63-7492fa11709a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.245232 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397fea8-4ede-4d38-af63-7492fa11709a admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.245977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397fea8-4ede-4d38-af63-7492fa11709a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:28:20.246289 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397fea8-4ede-4d38-af63-7492fa11709a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:20.247887 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397fea8-4ede-4d38-af63-7492fa11709a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:20.249719 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 786/1559] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:20.253327 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d96ae-b1cd-434d-a233-c3146267c33d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:20.260244 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d96ae-b1cd-434d-a233-c3146267c33d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:20.261097 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 774/1560] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:28:20.261975 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397fea8-4ede-4d38-af63-7492fa11709a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:20.271318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9d90e-ea55-4e1c-b860-2cf6e95f55ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:20.271984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9d90e-ea55-4e1c-b860-2cf6e95f55ed admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.272272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9d90e-ea55-4e1c-b860-2cf6e95f55ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.272554 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9d90e-ea55-4e1c-b860-2cf6e95f55ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.273294 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9d90e-ea55-4e1c-b860-2cf6e95f55ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:28:20.273629 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9d90e-ea55-4e1c-b860-2cf6e95f55ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:20.275804 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9d90e-ea55-4e1c-b860-2cf6e95f55ed admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:20.277021 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397fea8-4ede-4d38-af63-7492fa11709a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:20.282151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f1590-4039-4161-b097-fa6cacf40891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:20.283037 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f1590-4039-4161-b097-fa6cacf40891 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.283314 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f1590-4039-4161-b097-fa6cacf40891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.283586 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f1590-4039-4161-b097-fa6cacf40891 admin admin] PATH_INFO: `/v3/projects/89156f1867314be89e5108d5506b83a3/users/c66efdb1cc27467d85831617590c8b28/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.294649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 787/1561] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:28:20.294649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4bf3b6-8c8f-4c81-8ff5-59b04f27bd0c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.294649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4bf3b6-8c8f-4c81-8ff5-59b04f27bd0c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.294649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4bf3b6-8c8f-4c81-8ff5-59b04f27bd0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.298603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f1590-4039-4161-b097-fa6cacf40891 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89156f1867314be89e5108d5506b83a3, user_id=c66efdb1cc27467d85831617590c8b28, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:20.301269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f1590-4039-4161-b097-fa6cacf40891 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:20.308610 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0e4bf3b6-8c8f-4c81-8ff5-59b04f27bd0c None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:20.309919 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0e4bf3b6-8c8f-4c81-8ff5-59b04f27bd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:20.315115 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 775/1562] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:28:20 2022] PUT /identity/v3/projects/89156f1867314be89e5108d5506b83a3/users/c66efdb1cc27467d85831617590c8b28/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:20.316065 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f1590-4039-4161-b097-fa6cacf40891 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:20.362367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:20.371656 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:20.379888 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f1590-4039-4161-b097-fa6cacf40891 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:20.381524 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 788/1563] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:20.382435 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4bf3b6-8c8f-4c81-8ff5-59b04f27bd0c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:20.388891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aac776-0c47-498f-a1a4-45f97891cd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:20.389711 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aac776-0c47-498f-a1a4-45f97891cd93 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.390092 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aac776-0c47-498f-a1a4-45f97891cd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.390463 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aac776-0c47-498f-a1a4-45f97891cd93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.391257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aac776-0c47-498f-a1a4-45f97891cd93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:20.392102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4bf3b6-8c8f-4c81-8ff5-59b04f27bd0c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:20.393512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aac776-0c47-498f-a1a4-45f97891cd93 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:20.396627 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fe2d2-7559-4a0e-9410-ff356f2432ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:20.397276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fe2d2-7559-4a0e-9410-ff356f2432ff admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.397569 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fe2d2-7559-4a0e-9410-ff356f2432ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.397884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fe2d2-7559-4a0e-9410-ff356f2432ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.398604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fe2d2-7559-4a0e-9410-ff356f2432ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:28:20.398922 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fe2d2-7559-4a0e-9410-ff356f2432ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:20.401128 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fe2d2-7559-4a0e-9410-ff356f2432ff admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:20.412949 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 789/1564] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:28:20.414243 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8055cb-830b-4550-a176-1a2ca174cd75 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.414540 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8055cb-830b-4550-a176-1a2ca174cd75 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.414825 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8055cb-830b-4550-a176-1a2ca174cd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.426944 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 776/1565] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:28:20 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:20.427958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aac776-0c47-498f-a1a4-45f97891cd93 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:20.435786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-bc8055cb-830b-4550-a176-1a2ca174cd75 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:20.444058 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bc8055cb-830b-4550-a176-1a2ca174cd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:20.493044 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aac776-0c47-498f-a1a4-45f97891cd93 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:20.497115 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 790/1566] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:20.497972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8055cb-830b-4550-a176-1a2ca174cd75 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:20.501425 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207aee9-3d6f-470b-a59f-497e6cf8db1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:20.502130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207aee9-3d6f-470b-a59f-497e6cf8db1e admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.502456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207aee9-3d6f-470b-a59f-497e6cf8db1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.502747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207aee9-3d6f-470b-a59f-497e6cf8db1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.503481 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4207aee9-3d6f-470b-a59f-497e6cf8db1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:28:20.503793 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4207aee9-3d6f-470b-a59f-497e6cf8db1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:20.505432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4207aee9-3d6f-470b-a59f-497e6cf8db1e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:20.507048 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8055cb-830b-4550-a176-1a2ca174cd75 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:20.511816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea1d9d-9b2c-47a6-85bb-e3484c930d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:20.512453 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea1d9d-9b2c-47a6-85bb-e3484c930d25 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.512742 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea1d9d-9b2c-47a6-85bb-e3484c930d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.513023 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea1d9d-9b2c-47a6-85bb-e3484c930d25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.513737 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea1d9d-9b2c-47a6-85bb-e3484c930d25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:20.514748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-89ea1d9d-9b2c-47a6-85bb-e3484c930d25 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 18:28:20.514748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jun 30 18:28:20.514748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: Jun 30 18:28:20.515971 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea1d9d-9b2c-47a6-85bb-e3484c930d25 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:20.517883 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 777/1567] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:28:20.518667 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207aee9-3d6f-470b-a59f-497e6cf8db1e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:20.534965 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207aee9-3d6f-470b-a59f-497e6cf8db1e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:20.539676 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafb2f3-7f4a-4a56-81b3-7a5725a36d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:20.540328 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafb2f3-7f4a-4a56-81b3-7a5725a36d50 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.540863 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafb2f3-7f4a-4a56-81b3-7a5725a36d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.541003 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafb2f3-7f4a-4a56-81b3-7a5725a36d50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.541703 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fafb2f3-7f4a-4a56-81b3-7a5725a36d50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:20.546761 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 791/1568] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:28:20 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:20.547283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fafb2f3-7f4a-4a56-81b3-7a5725a36d50 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:20.547407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19a21f-ee13-43e9-8425-37291d5cd264 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.547407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19a21f-ee13-43e9-8425-37291d5cd264 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.547407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19a21f-ee13-43e9-8425-37291d5cd264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.563590 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7b19a21f-ee13-43e9-8425-37291d5cd264 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:20.564889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7b19a21f-ee13-43e9-8425-37291d5cd264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:20.576549 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 778/1569] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:28:20 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:20.576549 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafb2f3-7f4a-4a56-81b3-7a5725a36d50 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:20.625655 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 792/1570] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:20.626909 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19a21f-ee13-43e9-8425-37291d5cd264 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:20.636844 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19a21f-ee13-43e9-8425-37291d5cd264 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:20.638931 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafb2f3-7f4a-4a56-81b3-7a5725a36d50 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:20.641442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb153ad-4388-4b86-b005-b09497033492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:20.642284 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb153ad-4388-4b86-b005-b09497033492 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.642517 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb153ad-4388-4b86-b005-b09497033492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.642680 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb153ad-4388-4b86-b005-b09497033492 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.643470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb153ad-4388-4b86-b005-b09497033492 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:20.643801 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d4004-987f-4bfa-818f-ca27145817c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:20.644468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d4004-987f-4bfa-818f-ca27145817c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.644758 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d4004-987f-4bfa-818f-ca27145817c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.645042 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d4004-987f-4bfa-818f-ca27145817c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.645570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb153ad-4388-4b86-b005-b09497033492 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:20.645802 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d4004-987f-4bfa-818f-ca27145817c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:28:20.646117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d4004-987f-4bfa-818f-ca27145817c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:20.647760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d4004-987f-4bfa-818f-ca27145817c3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:20.660972 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 779/1571] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:28:20.661868 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d4004-987f-4bfa-818f-ca27145817c3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:20.671884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 793/1572] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:28:20 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:20.672707 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb153ad-4388-4b86-b005-b09497033492 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:20.678748 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d4004-987f-4bfa-818f-ca27145817c3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:20.710867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:20.719514 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:20.737909 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb153ad-4388-4b86-b005-b09497033492 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:20.738313 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc84d3f-ffbf-4696-af8d-1ef456b6dae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:20.739032 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc84d3f-ffbf-4696-af8d-1ef456b6dae9 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.739321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc84d3f-ffbf-4696-af8d-1ef456b6dae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.739597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc84d3f-ffbf-4696-af8d-1ef456b6dae9 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.740473 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc84d3f-ffbf-4696-af8d-1ef456b6dae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:20.741655 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-ddc84d3f-ffbf-4696-af8d-1ef456b6dae9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 18:28:20.741655 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jun 30 18:28:20.741655 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: Jun 30 18:28:20.742791 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b34f7-c31a-448f-b8df-d8a5946a66cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:20.742912 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc84d3f-ffbf-4696-af8d-1ef456b6dae9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:20.743534 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b34f7-c31a-448f-b8df-d8a5946a66cf admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.743933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b34f7-c31a-448f-b8df-d8a5946a66cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.744386 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b34f7-c31a-448f-b8df-d8a5946a66cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.745126 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b34f7-c31a-448f-b8df-d8a5946a66cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:28:20.745443 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b34f7-c31a-448f-b8df-d8a5946a66cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:20.747192 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b34f7-c31a-448f-b8df-d8a5946a66cf admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:20.759916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 794/1573] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:28:20.760723 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b34f7-c31a-448f-b8df-d8a5946a66cf admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:20.766049 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 780/1574] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:28:20 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:20.767018 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc84d3f-ffbf-4696-af8d-1ef456b6dae9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:20.770724 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b34f7-c31a-448f-b8df-d8a5946a66cf admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:20.775441 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6afbe-9ed3-456e-a8c2-86d49638a579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:20.776087 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6afbe-9ed3-456e-a8c2-86d49638a579 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.776378 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6afbe-9ed3-456e-a8c2-86d49638a579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.776661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6afbe-9ed3-456e-a8c2-86d49638a579 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.777170 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc84d3f-ffbf-4696-af8d-1ef456b6dae9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:20.777385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc6afbe-9ed3-456e-a8c2-86d49638a579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:20.779403 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc6afbe-9ed3-456e-a8c2-86d49638a579 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:20.781953 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c5921-bdfa-4ac3-8d4f-b5b089e7e688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:20.782611 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c5921-bdfa-4ac3-8d4f-b5b089e7e688 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.782882 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c5921-bdfa-4ac3-8d4f-b5b089e7e688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.783156 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c5921-bdfa-4ac3-8d4f-b5b089e7e688 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.783841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c5921-bdfa-4ac3-8d4f-b5b089e7e688 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:20.785790 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c5921-bdfa-4ac3-8d4f-b5b089e7e688 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:20.809567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 781/1575] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:28:20 2022] POST /identity/v3/users => generated 293 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:20.809837 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 795/1576] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:28:20 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:20.809948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c5921-bdfa-4ac3-8d4f-b5b089e7e688 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:20.810323 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6afbe-9ed3-456e-a8c2-86d49638a579 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:20.820043 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c5921-bdfa-4ac3-8d4f-b5b089e7e688 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:20.864753 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6afbe-9ed3-456e-a8c2-86d49638a579 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:20.868342 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61dc888-eeea-4b8b-b607-7f56de50460f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:20.869520 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61dc888-eeea-4b8b-b607-7f56de50460f service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.869838 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61dc888-eeea-4b8b-b607-7f56de50460f service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.870110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61dc888-eeea-4b8b-b607-7f56de50460f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.877812 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da001ff4-59b2-47aa-a2f5-44d38bfd75d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:20.878484 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da001ff4-59b2-47aa-a2f5-44d38bfd75d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.878772 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da001ff4-59b2-47aa-a2f5-44d38bfd75d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.879054 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da001ff4-59b2-47aa-a2f5-44d38bfd75d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.879742 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da001ff4-59b2-47aa-a2f5-44d38bfd75d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:20.881731 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da001ff4-59b2-47aa-a2f5-44d38bfd75d2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:20.903271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 782/1577] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:28:20 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:20.904144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da001ff4-59b2-47aa-a2f5-44d38bfd75d2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:20.913655 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da001ff4-59b2-47aa-a2f5-44d38bfd75d2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:20.919217 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3da61-ae90-42cb-8740-7d7d47820e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:20.919858 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3da61-ae90-42cb-8740-7d7d47820e5e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:20.920146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3da61-ae90-42cb-8740-7d7d47820e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:20.920432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3da61-ae90-42cb-8740-7d7d47820e5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:20.921118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f3da61-ae90-42cb-8740-7d7d47820e5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:20.923148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f3da61-ae90-42cb-8740-7d7d47820e5e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:20.936359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61dc888-eeea-4b8b-b607-7f56de50460f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:20.937844 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61dc888-eeea-4b8b-b607-7f56de50460f service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:20.948914 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 796/1578] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:20.949530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61dc888-eeea-4b8b-b607-7f56de50460f service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:20.954287 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 783/1579] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:28:20 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:20.954287 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3da61-ae90-42cb-8740-7d7d47820e5e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:20.958655 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61dc888-eeea-4b8b-b607-7f56de50460f service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:21.023449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3da61-ae90-42cb-8740-7d7d47820e5e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:21.026415 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de66fbe-1f32-42cf-ad23-61429251ea17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:21.027319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de66fbe-1f32-42cf-ad23-61429251ea17 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.028526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de66fbe-1f32-42cf-ad23-61429251ea17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.028935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de66fbe-1f32-42cf-ad23-61429251ea17 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.029667 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de66fbe-1f32-42cf-ad23-61429251ea17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:21.030274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d5ac47-0a23-4e09-bbe9-e0c83dcf4c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:21.030994 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5ac47-0a23-4e09-bbe9-e0c83dcf4c0c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.031308 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5ac47-0a23-4e09-bbe9-e0c83dcf4c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.031591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5ac47-0a23-4e09-bbe9-e0c83dcf4c0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.031892 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de66fbe-1f32-42cf-ad23-61429251ea17 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:21.032361 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d5ac47-0a23-4e09-bbe9-e0c83dcf4c0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:21.034115 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d5ac47-0a23-4e09-bbe9-e0c83dcf4c0c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:21.044764 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:21.047263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 784/1580] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:21.048679 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d5ac47-0a23-4e09-bbe9-e0c83dcf4c0c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:21.059278 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d5ac47-0a23-4e09-bbe9-e0c83dcf4c0c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:21.060833 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 797/1581] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:28:20 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:21.062185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de66fbe-1f32-42cf-ad23-61429251ea17 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:21.068616 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdcae27-53f8-4e38-9e92-ce33778ee3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:21.069291 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdcae27-53f8-4e38-9e92-ce33778ee3a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.069598 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdcae27-53f8-4e38-9e92-ce33778ee3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.069891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdcae27-53f8-4e38-9e92-ce33778ee3a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.071015 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdcae27-53f8-4e38-9e92-ce33778ee3a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:21.072683 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdcae27-53f8-4e38-9e92-ce33778ee3a3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:21.075263 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de66fbe-1f32-42cf-ad23-61429251ea17 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:21.080499 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8ef45-f055-4fbf-b80e-d60e4395dcf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:21.081065 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:21.081892 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8ef45-f055-4fbf-b80e-d60e4395dcf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.082182 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8ef45-f055-4fbf-b80e-d60e4395dcf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.082470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8ef45-f055-4fbf-b80e-d60e4395dcf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.083267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b8ef45-f055-4fbf-b80e-d60e4395dcf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:21.085162 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b8ef45-f055-4fbf-b80e-d60e4395dcf0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:21.097134 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 798/1582] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:21.098085 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8ef45-f055-4fbf-b80e-d60e4395dcf0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:21.110026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8ef45-f055-4fbf-b80e-d60e4395dcf0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:21.110749 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 785/1583] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:28:21 2022] POST /identity/v3/projects => generated 365 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:21.111806 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdcae27-53f8-4e38-9e92-ce33778ee3a3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:21.180983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdcae27-53f8-4e38-9e92-ce33778ee3a3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:21.183306 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e98f36-71e0-4e40-8c16-586b1dcc1fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:21.183988 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e98f36-71e0-4e40-8c16-586b1dcc1fc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.184278 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e98f36-71e0-4e40-8c16-586b1dcc1fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.184564 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e98f36-71e0-4e40-8c16-586b1dcc1fc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.185516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e98f36-71e0-4e40-8c16-586b1dcc1fc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:21.187543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e98f36-71e0-4e40-8c16-586b1dcc1fc8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:21.188346 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95109c5-8034-479b-96b1-844f1dcb6668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:21.189034 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95109c5-8034-479b-96b1-844f1dcb6668 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.189349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95109c5-8034-479b-96b1-844f1dcb6668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.189729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95109c5-8034-479b-96b1-844f1dcb6668 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.191030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95109c5-8034-479b-96b1-844f1dcb6668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:21.192789 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95109c5-8034-479b-96b1-844f1dcb6668 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:21.207824 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 786/1584] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:21.207824 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95109c5-8034-479b-96b1-844f1dcb6668 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:21.214154 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 799/1585] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:28:21 2022] POST /identity/v3/users => generated 282 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:21.214154 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e98f36-71e0-4e40-8c16-586b1dcc1fc8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:21.219066 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95109c5-8034-479b-96b1-844f1dcb6668 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:21.224264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8ab8d-633e-4099-9f27-8aaa542f4339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:21.224921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8ab8d-633e-4099-9f27-8aaa542f4339 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.225217 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e98f36-71e0-4e40-8c16-586b1dcc1fc8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:21.225334 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8ab8d-633e-4099-9f27-8aaa542f4339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.225926 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8ab8d-633e-4099-9f27-8aaa542f4339 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.226653 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8ab8d-633e-4099-9f27-8aaa542f4339 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:21.228788 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8ab8d-633e-4099-9f27-8aaa542f4339 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:21.235734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32f81d-45f0-4129-afa2-79062b886447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:21.236689 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32f81d-45f0-4129-afa2-79062b886447 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.237278 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32f81d-45f0-4129-afa2-79062b886447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.237760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32f81d-45f0-4129-afa2-79062b886447 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.239524 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32f81d-45f0-4129-afa2-79062b886447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:21.241900 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32f81d-45f0-4129-afa2-79062b886447 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:21.254039 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 787/1586] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:28:21 2022] POST /identity/v3/users => generated 289 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:21.254039 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8ab8d-633e-4099-9f27-8aaa542f4339 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:21.259322 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 800/1587] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:21.261006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32f81d-45f0-4129-afa2-79062b886447 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:21.269062 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8ab8d-633e-4099-9f27-8aaa542f4339 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:21.274840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b8529-caed-465c-a952-29c1b9093c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:21.275020 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32f81d-45f0-4129-afa2-79062b886447 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:21.275579 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b8529-caed-465c-a952-29c1b9093c25 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.275970 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b8529-caed-465c-a952-29c1b9093c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.276345 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b8529-caed-465c-a952-29c1b9093c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.277168 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b8529-caed-465c-a952-29c1b9093c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:21.281916 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b8529-caed-465c-a952-29c1b9093c25 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:21.285396 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255219f-48dc-41da-ba00-c8d87e81381d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:21.286282 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255219f-48dc-41da-ba00-c8d87e81381d admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.286850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255219f-48dc-41da-ba00-c8d87e81381d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.287222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255219f-48dc-41da-ba00-c8d87e81381d admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.288182 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255219f-48dc-41da-ba00-c8d87e81381d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:21.290021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255219f-48dc-41da-ba00-c8d87e81381d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:21.297786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 788/1588] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:21.298982 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b8529-caed-465c-a952-29c1b9093c25 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:21.305020 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 801/1589] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:21.305020 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255219f-48dc-41da-ba00-c8d87e81381d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:21.312457 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b8529-caed-465c-a952-29c1b9093c25 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:21.318494 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74607ea-427d-488a-b552-1e3ed0988daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:21.318657 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255219f-48dc-41da-ba00-c8d87e81381d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:21.319345 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74607ea-427d-488a-b552-1e3ed0988daf admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.319784 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74607ea-427d-488a-b552-1e3ed0988daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.320167 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74607ea-427d-488a-b552-1e3ed0988daf admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.320974 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74607ea-427d-488a-b552-1e3ed0988daf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:21.323148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74607ea-427d-488a-b552-1e3ed0988daf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:21.323610 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c9812-eebe-4d6e-8afe-a976192523f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:21.324392 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c9812-eebe-4d6e-8afe-a976192523f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.324689 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c9812-eebe-4d6e-8afe-a976192523f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.325081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c9812-eebe-4d6e-8afe-a976192523f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.325857 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c9812-eebe-4d6e-8afe-a976192523f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:21.327680 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c9812-eebe-4d6e-8afe-a976192523f6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:21.348328 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 789/1590] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:28:21 2022] POST /identity/v3/users => generated 283 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:21.349158 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74607ea-427d-488a-b552-1e3ed0988daf admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:21.350377 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 802/1591] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:21.351151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c9812-eebe-4d6e-8afe-a976192523f6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:21.360656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74607ea-427d-488a-b552-1e3ed0988daf admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:21.365312 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8449b-6f81-4597-8278-b87d0d482d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:21.366013 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8449b-6f81-4597-8278-b87d0d482d43 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.366292 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8449b-6f81-4597-8278-b87d0d482d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.366564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8449b-6f81-4597-8278-b87d0d482d43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.367413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8449b-6f81-4597-8278-b87d0d482d43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:21.369259 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8449b-6f81-4597-8278-b87d0d482d43 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:21.371889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c9812-eebe-4d6e-8afe-a976192523f6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:21.376344 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ee60f-c427-439a-9467-bf34576d97d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:21.377021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ee60f-c427-439a-9467-bf34576d97d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.377551 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ee60f-c427-439a-9467-bf34576d97d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.378040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ee60f-c427-439a-9467-bf34576d97d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.378777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ee60f-c427-439a-9467-bf34576d97d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:21.380553 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ee60f-c427-439a-9467-bf34576d97d7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:21.381748 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 790/1592] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:21.386497 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8449b-6f81-4597-8278-b87d0d482d43 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:21.393280 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 803/1593] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:21.394206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ee60f-c427-439a-9467-bf34576d97d7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:21.400423 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8449b-6f81-4597-8278-b87d0d482d43 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:21.407436 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ee60f-c427-439a-9467-bf34576d97d7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:21.408850 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8395b4-7c05-4599-b0b4-05936c9e928f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:21.410488 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8395b4-7c05-4599-b0b4-05936c9e928f admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.410907 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8395b4-7c05-4599-b0b4-05936c9e928f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.411320 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8395b4-7c05-4599-b0b4-05936c9e928f admin admin] PATH_INFO: `/v3/projects/8670df9b372441429766eb047c891e92/users/e146fbe0451c44389e57276b4af5a55e/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.413645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c4e2b-3da8-4176-9027-94c3132cc350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:21.414326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c4e2b-3da8-4176-9027-94c3132cc350 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.415546 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c4e2b-3da8-4176-9027-94c3132cc350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.416882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c4e2b-3da8-4176-9027-94c3132cc350 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.417688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c4e2b-3da8-4176-9027-94c3132cc350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:21.419579 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c4e2b-3da8-4176-9027-94c3132cc350 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:21.432963 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8395b4-7c05-4599-b0b4-05936c9e928f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8670df9b372441429766eb047c891e92, user_id=e146fbe0451c44389e57276b4af5a55e, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:21.433222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 804/1594] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:21.433893 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c4e2b-3da8-4176-9027-94c3132cc350 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:21.437602 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8395b4-7c05-4599-b0b4-05936c9e928f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:21.445395 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c4e2b-3da8-4176-9027-94c3132cc350 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:21.452324 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a0abce-ea6d-47f4-8fe8-b9b1f0c4dc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:21.453269 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a0abce-ea6d-47f4-8fe8-b9b1f0c4dc21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.453716 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a0abce-ea6d-47f4-8fe8-b9b1f0c4dc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.453827 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a0abce-ea6d-47f4-8fe8-b9b1f0c4dc21 admin admin] PATH_INFO: `/v3/projects/f643222ab6194191979d2e884f5ed75a/users/f13e9e1bf5084d08919845767c70c3f7/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.460494 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 791/1595] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:28:21 2022] PUT /identity/v3/projects/8670df9b372441429766eb047c891e92/users/e146fbe0451c44389e57276b4af5a55e/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:21.460494 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8395b4-7c05-4599-b0b4-05936c9e928f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:21.480242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a0abce-ea6d-47f4-8fe8-b9b1f0c4dc21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f643222ab6194191979d2e884f5ed75a, user_id=f13e9e1bf5084d08919845767c70c3f7, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:21.481614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-55a0abce-ea6d-47f4-8fe8-b9b1f0c4dc21 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 18:28:21.481614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jun 30 18:28:21.481614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: Jun 30 18:28:21.484040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a0abce-ea6d-47f4-8fe8-b9b1f0c4dc21 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:21.498661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 805/1596] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:28:21 2022] PUT /identity/v3/projects/f643222ab6194191979d2e884f5ed75a/users/f13e9e1bf5084d08919845767c70c3f7/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:21.502861 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a0abce-ea6d-47f4-8fe8-b9b1f0c4dc21 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:21.537709 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8395b4-7c05-4599-b0b4-05936c9e928f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:21.547067 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:21.567742 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:21.573626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a0abce-ea6d-47f4-8fe8-b9b1f0c4dc21 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:21.579929 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56705e01-2186-4d81-9030-116a04d0ee8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:21.580963 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56705e01-2186-4d81-9030-116a04d0ee8a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.581190 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56705e01-2186-4d81-9030-116a04d0ee8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.581708 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56705e01-2186-4d81-9030-116a04d0ee8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.582548 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56705e01-2186-4d81-9030-116a04d0ee8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:21.584285 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56705e01-2186-4d81-9030-116a04d0ee8a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:21.595619 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 806/1597] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:21.596742 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56705e01-2186-4d81-9030-116a04d0ee8a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:21.607334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56705e01-2186-4d81-9030-116a04d0ee8a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:21.612585 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7f53b-21b5-4416-a685-bf8e924ca29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:21.613109 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd5f14c-7097-4359-aa40-b3cf5b56bef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:21.613925 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5f14c-7097-4359-aa40-b3cf5b56bef2 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.614038 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7f53b-21b5-4416-a685-bf8e924ca29e admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.614282 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7f53b-21b5-4416-a685-bf8e924ca29e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.614560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7f53b-21b5-4416-a685-bf8e924ca29e admin admin] PATH_INFO: `/v3/projects/6a127ed10b0a465aa487242a344defa0/users/4f262bfdf0d54e76a77e8ce8cda137dc/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.614773 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5f14c-7097-4359-aa40-b3cf5b56bef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.615221 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5f14c-7097-4359-aa40-b3cf5b56bef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.616386 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd5f14c-7097-4359-aa40-b3cf5b56bef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:21.621189 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd5f14c-7097-4359-aa40-b3cf5b56bef2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:21.632352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 807/1598] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:21.633314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd5f14c-7097-4359-aa40-b3cf5b56bef2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:21.638869 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7f53b-21b5-4416-a685-bf8e924ca29e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a127ed10b0a465aa487242a344defa0, user_id=4f262bfdf0d54e76a77e8ce8cda137dc, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:21.641911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7f53b-21b5-4416-a685-bf8e924ca29e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:21.645422 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd5f14c-7097-4359-aa40-b3cf5b56bef2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:21.652299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec4f3a-1822-42e9-993b-5e563c12e196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:21.653232 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec4f3a-1822-42e9-993b-5e563c12e196 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.653565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec4f3a-1822-42e9-993b-5e563c12e196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.654223 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec4f3a-1822-42e9-993b-5e563c12e196 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.655005 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec4f3a-1822-42e9-993b-5e563c12e196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:21.657078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec4f3a-1822-42e9-993b-5e563c12e196 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:21.658872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 792/1599] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:28:21 2022] PUT /identity/v3/projects/6a127ed10b0a465aa487242a344defa0/users/4f262bfdf0d54e76a77e8ce8cda137dc/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:21.660060 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7f53b-21b5-4416-a685-bf8e924ca29e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:21.685957 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 808/1600] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:21.690708 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ba112-a3fb-41d2-83be-b77650bb5a3c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.691733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ba112-a3fb-41d2-83be-b77650bb5a3c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.691951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ba112-a3fb-41d2-83be-b77650bb5a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.725895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ea5ba112-a3fb-41d2-83be-b77650bb5a3c None None] MFA Rules not processed for user `e146fbe0451c44389e57276b4af5a55e`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:21.727216 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ea5ba112-a3fb-41d2-83be-b77650bb5a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:21.745969 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7f53b-21b5-4416-a685-bf8e924ca29e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:21.751010 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eaf406-4f9c-42b1-b4c2-5f2c46a751ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:21.752403 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eaf406-4f9c-42b1-b4c2-5f2c46a751ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.752697 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eaf406-4f9c-42b1-b4c2-5f2c46a751ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.752981 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eaf406-4f9c-42b1-b4c2-5f2c46a751ff admin admin] PATH_INFO: `/v3/projects/97d3ba408c3c4cc78d2a8e983869e11e/users/31234f42e4454706aa9295881d0abe0a/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.773291 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eaf406-4f9c-42b1-b4c2-5f2c46a751ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=97d3ba408c3c4cc78d2a8e983869e11e, user_id=31234f42e4454706aa9295881d0abe0a, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:21.778292 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eaf406-4f9c-42b1-b4c2-5f2c46a751ff admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:21.809881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 793/1601] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:28:21 2022] PUT /identity/v3/projects/97d3ba408c3c4cc78d2a8e983869e11e/users/31234f42e4454706aa9295881d0abe0a/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:21.809881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead23386-0f1d-40ea-bad5-68723890c884 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.809881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead23386-0f1d-40ea-bad5-68723890c884 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.809881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead23386-0f1d-40ea-bad5-68723890c884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.827564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ead23386-0f1d-40ea-bad5-68723890c884 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:21.827787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ead23386-0f1d-40ea-bad5-68723890c884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:21.828275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 809/1602] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:21.829414 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ba112-a3fb-41d2-83be-b77650bb5a3c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:21.843157 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ba112-a3fb-41d2-83be-b77650bb5a3c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:21.913080 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93162fde-d538-4e1e-b808-c81a0efb2f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:21.913836 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93162fde-d538-4e1e-b808-c81a0efb2f07 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.914226 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93162fde-d538-4e1e-b808-c81a0efb2f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.914527 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93162fde-d538-4e1e-b808-c81a0efb2f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.915307 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93162fde-d538-4e1e-b808-c81a0efb2f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:21.917044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93162fde-d538-4e1e-b808-c81a0efb2f07 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:21.922121 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 794/1603] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:21.923376 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead23386-0f1d-40ea-bad5-68723890c884 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:21.936470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 810/1604] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:21.936470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6680d7-07ff-4bf7-9ede-9763338342e7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.936470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6680d7-07ff-4bf7-9ede-9763338342e7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.936470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6680d7-07ff-4bf7-9ede-9763338342e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.939489 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead23386-0f1d-40ea-bad5-68723890c884 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:21.948173 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b77f8e-02b1-4c41-9379-0ef3e5bd1ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:21.949851 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b77f8e-02b1-4c41-9379-0ef3e5bd1ac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:21.950244 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b77f8e-02b1-4c41-9379-0ef3e5bd1ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:21.950642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b77f8e-02b1-4c41-9379-0ef3e5bd1ac2 admin admin] PATH_INFO: `/v3/projects/58e2901bf4f847fcbdf60af88dd29207/users/13907c48e3ea4b158b4b49fb7fbae2ee/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:21.970424 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0f6680d7-07ff-4bf7-9ede-9763338342e7 None None] MFA Rules not processed for user `4f262bfdf0d54e76a77e8ce8cda137dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:21.971843 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0f6680d7-07ff-4bf7-9ede-9763338342e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:21.973611 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b77f8e-02b1-4c41-9379-0ef3e5bd1ac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58e2901bf4f847fcbdf60af88dd29207, user_id=13907c48e3ea4b158b4b49fb7fbae2ee, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:21.976459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b77f8e-02b1-4c41-9379-0ef3e5bd1ac2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:21.978977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:21.994722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 795/1605] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:28:21 2022] PUT /identity/v3/projects/58e2901bf4f847fcbdf60af88dd29207/users/13907c48e3ea4b158b4b49fb7fbae2ee/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:21.995745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b77f8e-02b1-4c41-9379-0ef3e5bd1ac2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:22.036420 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:22.073645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 811/1606] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:22.075015 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72485005-f78a-40b4-902b-9124648ae319 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:22.075314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72485005-f78a-40b4-902b-9124648ae319 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:22.075607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72485005-f78a-40b4-902b-9124648ae319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:22.083931 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b77f8e-02b1-4c41-9379-0ef3e5bd1ac2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:22.089008 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9a1f9-5d9b-40bd-bf1a-239203d424b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:22.090222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9a1f9-5d9b-40bd-bf1a-239203d424b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:22.090692 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9a1f9-5d9b-40bd-bf1a-239203d424b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:22.091014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9a1f9-5d9b-40bd-bf1a-239203d424b0 admin admin] PATH_INFO: `/v3/projects/e96b506102ab4528956023c5e0f3dbcf/users/db9762ad11e24246aefe2905eafe83f1/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:22.109071 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d9a1f9-5d9b-40bd-bf1a-239203d424b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e96b506102ab4528956023c5e0f3dbcf, user_id=db9762ad11e24246aefe2905eafe83f1, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:22.111935 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d9a1f9-5d9b-40bd-bf1a-239203d424b0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:22.113624 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-72485005-f78a-40b4-902b-9124648ae319 None None] MFA Rules not processed for user `31234f42e4454706aa9295881d0abe0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:22.115824 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-72485005-f78a-40b4-902b-9124648ae319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:22.131304 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 796/1607] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:28:21 2022] PUT /identity/v3/projects/e96b506102ab4528956023c5e0f3dbcf/users/db9762ad11e24246aefe2905eafe83f1/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:22.131304 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9a1f9-5d9b-40bd-bf1a-239203d424b0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:22.212673 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 812/1608] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:22.216559 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9a1f9-5d9b-40bd-bf1a-239203d424b0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:22.217294 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72485005-f78a-40b4-902b-9124648ae319 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:22.218709 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-72485005-f78a-40b4-902b-9124648ae319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:22.223840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ef35d-b726-47a8-8084-56450097295d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:22.225153 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ef35d-b726-47a8-8084-56450097295d admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:22.225544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ef35d-b726-47a8-8084-56450097295d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:22.225956 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ef35d-b726-47a8-8084-56450097295d admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:22.227081 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ef35d-b726-47a8-8084-56450097295d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:22.230539 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ef35d-b726-47a8-8084-56450097295d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:22.267394 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 797/1609] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:28:22 2022] POST /identity/v3/projects => generated 375 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:22.268498 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ef35d-b726-47a8-8084-56450097295d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:22.297472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72485005-f78a-40b4-902b-9124648ae319 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:22.354715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ef35d-b726-47a8-8084-56450097295d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:22.356597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc56d3-eabb-42c7-8589-2db2f1b535b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:22.357377 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc56d3-eabb-42c7-8589-2db2f1b535b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:22.357827 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc56d3-eabb-42c7-8589-2db2f1b535b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:22.358269 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc56d3-eabb-42c7-8589-2db2f1b535b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:22.363038 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187242eb-9b60-43f8-9cfc-54c96059e94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:22.364156 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187242eb-9b60-43f8-9cfc-54c96059e94e admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:22.364626 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187242eb-9b60-43f8-9cfc-54c96059e94e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:22.365079 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187242eb-9b60-43f8-9cfc-54c96059e94e admin admin] PATH_INFO: `/v3/projects/0a6721a373b04c71bf1f725123d69891/users/58181dfcd0ff4abea8494655d837c4d8/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:22.371367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:22.373679 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc56d3-eabb-42c7-8589-2db2f1b535b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:22.376547 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc56d3-eabb-42c7-8589-2db2f1b535b4 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:22.390351 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187242eb-9b60-43f8-9cfc-54c96059e94e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a6721a373b04c71bf1f725123d69891, user_id=58181dfcd0ff4abea8494655d837c4d8, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:22.392188 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 813/1610] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:22.393736 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772991e-7195-49ed-b397-f010d6d318bb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:22.394209 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772991e-7195-49ed-b397-f010d6d318bb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:22.394654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772991e-7195-49ed-b397-f010d6d318bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:22.408182 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187242eb-9b60-43f8-9cfc-54c96059e94e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:22.432677 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e772991e-7195-49ed-b397-f010d6d318bb None None] MFA Rules not processed for user `13907c48e3ea4b158b4b49fb7fbae2ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:22.436029 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e772991e-7195-49ed-b397-f010d6d318bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:22.440554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 798/1611] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:28:22 2022] PUT /identity/v3/projects/0a6721a373b04c71bf1f725123d69891/users/58181dfcd0ff4abea8494655d837c4d8/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:22.442464 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6901d-e3a2-4645-a33e-1f4ed1938738 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:22.444236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6901d-e3a2-4645-a33e-1f4ed1938738 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:22.444624 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6901d-e3a2-4645-a33e-1f4ed1938738 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:22.468027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 799/1612] 10.222.0.33 () {54 vars in 961 bytes} [Thu Jun 30 18:28:22 2022] GET /identity => generated 270 bytes in 27 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:28:22.474660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d90df1-0618-419e-be17-e4992719a114 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:22.475847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d90df1-0618-419e-be17-e4992719a114 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:22.477915 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d90df1-0618-419e-be17-e4992719a114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:22.479867 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:22.539476 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-48d90df1-0618-419e-be17-e4992719a114 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:22.540853 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-48d90df1-0618-419e-be17-e4992719a114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:22.551138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 814/1613] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:22 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:22.555123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772991e-7195-49ed-b397-f010d6d318bb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:22.637869 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 800/1614] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:22.638411 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d90df1-0618-419e-be17-e4992719a114 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:22.660076 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d90df1-0618-419e-be17-e4992719a114 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:22.672445 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3518497-d501-4e96-a857-3a22ed926fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:22.673841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3518497-d501-4e96-a857-3a22ed926fce admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:22.675293 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3518497-d501-4e96-a857-3a22ed926fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:22.676725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3518497-d501-4e96-a857-3a22ed926fce admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:22.678408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3518497-d501-4e96-a857-3a22ed926fce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:22.682986 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3518497-d501-4e96-a857-3a22ed926fce admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:22.688346 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772991e-7195-49ed-b397-f010d6d318bb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:22.694643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860afa66-5326-4bb8-b417-fffe6aa13f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:22.695784 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860afa66-5326-4bb8-b417-fffe6aa13f2b admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:22.696253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860afa66-5326-4bb8-b417-fffe6aa13f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:22.696510 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860afa66-5326-4bb8-b417-fffe6aa13f2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:22.697418 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860afa66-5326-4bb8-b417-fffe6aa13f2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:22.701917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860afa66-5326-4bb8-b417-fffe6aa13f2b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:22.746016 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 801/1615] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:28:22 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:22.752168 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673de5a-b005-4ce3-b83c-38e4f25d8ebb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:22.752469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673de5a-b005-4ce3-b83c-38e4f25d8ebb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:22.752750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673de5a-b005-4ce3-b83c-38e4f25d8ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:22.757248 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 815/1616] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:28:22 2022] POST /identity/v3/projects => generated 383 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:22.760347 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7180674-dca2-45e6-bae5-1702f9a64942 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:22.760736 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7180674-dca2-45e6-bae5-1702f9a64942 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:22.761219 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7180674-dca2-45e6-bae5-1702f9a64942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:22.789654 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a673de5a-b005-4ce3-b83c-38e4f25d8ebb None None] MFA Rules not processed for user `58181dfcd0ff4abea8494655d837c4d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:22.790992 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a673de5a-b005-4ce3-b83c-38e4f25d8ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:22.797189 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f7180674-dca2-45e6-bae5-1702f9a64942 None None] MFA Rules not processed for user `b8751a4e92174d18b7b138392d2579a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:22.798529 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f7180674-dca2-45e6-bae5-1702f9a64942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:22.922484 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 802/1617] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:22 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:22.924407 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673de5a-b005-4ce3-b83c-38e4f25d8ebb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:22.961787 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 816/1618] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Jun 30 18:28:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:22.962839 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7180674-dca2-45e6-bae5-1702f9a64942 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:23.010497 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673de5a-b005-4ce3-b83c-38e4f25d8ebb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:23.016095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741a244-2fa1-4458-a306-dbd58eaf977a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:23.017356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741a244-2fa1-4458-a306-dbd58eaf977a service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:23.017679 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741a244-2fa1-4458-a306-dbd58eaf977a service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:23.017967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741a244-2fa1-4458-a306-dbd58eaf977a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:23.047896 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:23.051879 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:23.072595 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7180674-dca2-45e6-bae5-1702f9a64942 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:23.076526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebd81b-6d5f-4e9c-a42e-dd788097e9da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:23.077621 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebd81b-6d5f-4e9c-a42e-dd788097e9da service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:23.078020 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebd81b-6d5f-4e9c-a42e-dd788097e9da service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:23.078406 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebd81b-6d5f-4e9c-a42e-dd788097e9da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:23.115106 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741a244-2fa1-4458-a306-dbd58eaf977a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:23.118804 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741a244-2fa1-4458-a306-dbd58eaf977a service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:23.130553 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 803/1619] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:22 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:23.132353 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741a244-2fa1-4458-a306-dbd58eaf977a service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:23.149212 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ebd81b-6d5f-4e9c-a42e-dd788097e9da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:23.151983 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ebd81b-6d5f-4e9c-a42e-dd788097e9da service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:23.165203 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 817/1620] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:23.166552 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebd81b-6d5f-4e9c-a42e-dd788097e9da service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:23.186496 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebd81b-6d5f-4e9c-a42e-dd788097e9da service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:23.199466 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b3dbc-e589-4cf1-abb5-5f13c2d69f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:23.200650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b3dbc-e589-4cf1-abb5-5f13c2d69f8f admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:23.201145 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b3dbc-e589-4cf1-abb5-5f13c2d69f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:23.201522 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b3dbc-e589-4cf1-abb5-5f13c2d69f8f admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:23.202378 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b3dbc-e589-4cf1-abb5-5f13c2d69f8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:23.206534 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b3dbc-e589-4cf1-abb5-5f13c2d69f8f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:23.256329 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 818/1621] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:28:23 2022] POST /identity/v3/users => generated 298 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:23.257368 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b3dbc-e589-4cf1-abb5-5f13c2d69f8f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:23.279273 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b3dbc-e589-4cf1-abb5-5f13c2d69f8f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:23.280627 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741a244-2fa1-4458-a306-dbd58eaf977a service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:23.287748 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5d166-63f1-4032-bddd-e5fd94cd74f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:23.289320 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5d166-63f1-4032-bddd-e5fd94cd74f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:23.289808 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5d166-63f1-4032-bddd-e5fd94cd74f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:23.290292 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5d166-63f1-4032-bddd-e5fd94cd74f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:23.292059 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5d166-63f1-4032-bddd-e5fd94cd74f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:23.292847 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6063ae-e70e-4a7d-8711-ff922f05a329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:23.294751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6063ae-e70e-4a7d-8711-ff922f05a329 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:23.295185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6063ae-e70e-4a7d-8711-ff922f05a329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:23.296287 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6063ae-e70e-4a7d-8711-ff922f05a329 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:23.296516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5d166-63f1-4032-bddd-e5fd94cd74f1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:23.297930 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6063ae-e70e-4a7d-8711-ff922f05a329 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:23.301542 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6063ae-e70e-4a7d-8711-ff922f05a329 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:23.313236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 804/1622] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:23 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:23.314634 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5d166-63f1-4032-bddd-e5fd94cd74f1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:23.339768 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5d166-63f1-4032-bddd-e5fd94cd74f1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:23.347164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208b9bd-5476-4f0f-8ec2-cab97f2d0dbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:23.348084 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208b9bd-5476-4f0f-8ec2-cab97f2d0dbb service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:23.348509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208b9bd-5476-4f0f-8ec2-cab97f2d0dbb service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:23.348927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208b9bd-5476-4f0f-8ec2-cab97f2d0dbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:23.354264 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 819/1623] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:28:23 2022] POST /identity/v3/projects => generated 363 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:23.355372 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6063ae-e70e-4a7d-8711-ff922f05a329 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:23.512608 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6063ae-e70e-4a7d-8711-ff922f05a329 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:23.519362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13feca07-1846-4f34-87ed-7b5eced1056a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:23.521489 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13feca07-1846-4f34-87ed-7b5eced1056a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:23.521831 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13feca07-1846-4f34-87ed-7b5eced1056a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:23.522116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13feca07-1846-4f34-87ed-7b5eced1056a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:23.522922 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13feca07-1846-4f34-87ed-7b5eced1056a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:23.523625 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208b9bd-5476-4f0f-8ec2-cab97f2d0dbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:23.526420 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208b9bd-5476-4f0f-8ec2-cab97f2d0dbb service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:23.526704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13feca07-1846-4f34-87ed-7b5eced1056a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:23.562918 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 820/1624] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:23 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:23.565844 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13feca07-1846-4f34-87ed-7b5eced1056a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:23.573950 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 805/1625] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:23.574812 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208b9bd-5476-4f0f-8ec2-cab97f2d0dbb service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:23.596423 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208b9bd-5476-4f0f-8ec2-cab97f2d0dbb service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:23.603120 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b59d5-5c57-4010-8ad9-ca77a1c2845f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:23.604402 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b59d5-5c57-4010-8ad9-ca77a1c2845f admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:23.604587 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13feca07-1846-4f34-87ed-7b5eced1056a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:23.604934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b59d5-5c57-4010-8ad9-ca77a1c2845f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:23.605321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b59d5-5c57-4010-8ad9-ca77a1c2845f admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:23.606495 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b59d5-5c57-4010-8ad9-ca77a1c2845f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:23.609314 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b59d5-5c57-4010-8ad9-ca77a1c2845f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:23.616196 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a528c5-f8f3-4c18-adcb-5a8f3e4a6be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:23.619569 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a528c5-f8f3-4c18-adcb-5a8f3e4a6be8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:23.619569 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a528c5-f8f3-4c18-adcb-5a8f3e4a6be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:23.620995 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a528c5-f8f3-4c18-adcb-5a8f3e4a6be8 admin admin] PATH_INFO: `/v3/projects/9dd6d8b27c1b4d3f9a2fae94f516ed24/users/f510a5006aa144bbaafb33feea40f83b/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:23.645350 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 806/1626] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:28:23 2022] POST /identity/v3/users => generated 288 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:23.646506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b59d5-5c57-4010-8ad9-ca77a1c2845f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:23.663686 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a528c5-f8f3-4c18-adcb-5a8f3e4a6be8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dd6d8b27c1b4d3f9a2fae94f516ed24, user_id=f510a5006aa144bbaafb33feea40f83b, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:23.672413 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a528c5-f8f3-4c18-adcb-5a8f3e4a6be8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:23.674176 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b59d5-5c57-4010-8ad9-ca77a1c2845f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:23.684133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9af351-0719-4163-900e-93526bc40f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:23.685602 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9af351-0719-4163-900e-93526bc40f06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:23.686025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9af351-0719-4163-900e-93526bc40f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:23.686410 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9af351-0719-4163-900e-93526bc40f06 admin admin] PATH_INFO: `/v3/projects/ecbee9bc0f7749b386ad738063690e0c/users/df74872b1cad46fca19b9e37035f38ef/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:23.709666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 821/1627] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:28:23 2022] PUT /identity/v3/projects/9dd6d8b27c1b4d3f9a2fae94f516ed24/users/f510a5006aa144bbaafb33feea40f83b/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:23.713689 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9af351-0719-4163-900e-93526bc40f06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecbee9bc0f7749b386ad738063690e0c, user_id=df74872b1cad46fca19b9e37035f38ef, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:23.713931 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a528c5-f8f3-4c18-adcb-5a8f3e4a6be8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:23.721057 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9af351-0719-4163-900e-93526bc40f06 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:23.758872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 807/1628] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:28:23 2022] PUT /identity/v3/projects/ecbee9bc0f7749b386ad738063690e0c/users/df74872b1cad46fca19b9e37035f38ef/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:23.767286 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9af351-0719-4163-900e-93526bc40f06 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:23.783859 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:23.786168 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:23.903804 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a528c5-f8f3-4c18-adcb-5a8f3e4a6be8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:23.924167 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129001b-c78e-4ebc-a390-9998a63fb8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:23.925312 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129001b-c78e-4ebc-a390-9998a63fb8f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:23.925867 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129001b-c78e-4ebc-a390-9998a63fb8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:23.926426 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129001b-c78e-4ebc-a390-9998a63fb8f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:23.927327 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129001b-c78e-4ebc-a390-9998a63fb8f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:23.929344 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9af351-0719-4163-900e-93526bc40f06 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:23.929944 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129001b-c78e-4ebc-a390-9998a63fb8f1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:23.936353 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0226f2-6523-42c5-923c-ee0fbe3a5793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:23.938505 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0226f2-6523-42c5-923c-ee0fbe3a5793 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:23.938862 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0226f2-6523-42c5-923c-ee0fbe3a5793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:23.939213 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0226f2-6523-42c5-923c-ee0fbe3a5793 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:23.940674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0226f2-6523-42c5-923c-ee0fbe3a5793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:23.942471 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0226f2-6523-42c5-923c-ee0fbe3a5793 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:23.966931 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 808/1629] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:23 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:23.967679 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0226f2-6523-42c5-923c-ee0fbe3a5793 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:23.995196 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 822/1630] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:23 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:24.006327 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129001b-c78e-4ebc-a390-9998a63fb8f1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:24.024262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0226f2-6523-42c5-923c-ee0fbe3a5793 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:24.042748 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48673a3-d194-4966-89f8-835e9788f658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:24.043204 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48673a3-d194-4966-89f8-835e9788f658 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:24.043759 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48673a3-d194-4966-89f8-835e9788f658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:24.044831 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48673a3-d194-4966-89f8-835e9788f658 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:24.044992 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48673a3-d194-4966-89f8-835e9788f658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:24.047880 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48673a3-d194-4966-89f8-835e9788f658 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:24.055809 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129001b-c78e-4ebc-a390-9998a63fb8f1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:24.078627 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31da9b2-e2ab-47c7-96c7-1f59587426b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:24.079693 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31da9b2-e2ab-47c7-96c7-1f59587426b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:24.080731 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31da9b2-e2ab-47c7-96c7-1f59587426b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:24.081090 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31da9b2-e2ab-47c7-96c7-1f59587426b0 admin admin] PATH_INFO: `/v3/projects/9dd6d8b27c1b4d3f9a2fae94f516ed24/users/f510a5006aa144bbaafb33feea40f83b/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:24.081610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 809/1631] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:23 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:24.084950 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48673a3-d194-4966-89f8-835e9788f658 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:24.090501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31da9b2-e2ab-47c7-96c7-1f59587426b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dd6d8b27c1b4d3f9a2fae94f516ed24, user_id=f510a5006aa144bbaafb33feea40f83b, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:24.093342 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31da9b2-e2ab-47c7-96c7-1f59587426b0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:24.109400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48673a3-d194-4966-89f8-835e9788f658 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:24.124139 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 823/1632] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:28:23 2022] PUT /identity/v3/projects/9dd6d8b27c1b4d3f9a2fae94f516ed24/users/f510a5006aa144bbaafb33feea40f83b/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:24.133633 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31da9b2-e2ab-47c7-96c7-1f59587426b0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:24.303184 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d371f-c520-46ef-93b2-b99735a5dbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:24.304812 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d371f-c520-46ef-93b2-b99735a5dbbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:24.306803 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d371f-c520-46ef-93b2-b99735a5dbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:24.307073 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d371f-c520-46ef-93b2-b99735a5dbbe admin admin] PATH_INFO: `/v3/projects/4a572f8d3641497f9f9be8866b5c89a1/users/659d03f3fe734f6985a6a84feba5b272/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:24.335820 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31da9b2-e2ab-47c7-96c7-1f59587426b0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:24.342919 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6d371f-c520-46ef-93b2-b99735a5dbbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a572f8d3641497f9f9be8866b5c89a1, user_id=659d03f3fe734f6985a6a84feba5b272, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:24.345243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6d371f-c520-46ef-93b2-b99735a5dbbe admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:24.351963 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fae940-1133-4386-aab0-611abfd84354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:24.354378 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fae940-1133-4386-aab0-611abfd84354 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:24.354911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fae940-1133-4386-aab0-611abfd84354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:24.355327 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fae940-1133-4386-aab0-611abfd84354 admin admin] PATH_INFO: `/v3/projects/ecbee9bc0f7749b386ad738063690e0c/users/df74872b1cad46fca19b9e37035f38ef/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:24.359383 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fae940-1133-4386-aab0-611abfd84354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecbee9bc0f7749b386ad738063690e0c, user_id=df74872b1cad46fca19b9e37035f38ef, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:24.369359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fae940-1133-4386-aab0-611abfd84354 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:24.384102 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 810/1633] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:28:24 2022] PUT /identity/v3/projects/4a572f8d3641497f9f9be8866b5c89a1/users/659d03f3fe734f6985a6a84feba5b272/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:24.387542 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632c21d-8bc6-402e-8766-07f6ed3a2742 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:24.388469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632c21d-8bc6-402e-8766-07f6ed3a2742 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:24.388874 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632c21d-8bc6-402e-8766-07f6ed3a2742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:24.414857 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 824/1634] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:28:24 2022] PUT /identity/v3/projects/ecbee9bc0f7749b386ad738063690e0c/users/df74872b1cad46fca19b9e37035f38ef/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:24.417473 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fb9b8-7aa4-407a-9eda-8b7920259261 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:24.426353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fb9b8-7aa4-407a-9eda-8b7920259261 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:24.426778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fb9b8-7aa4-407a-9eda-8b7920259261 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:24.434501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 825/1635] 10.222.0.33 () {54 vars in 912 bytes} [Thu Jun 30 18:28:24 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:28:24.436183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fb9b8-7aa4-407a-9eda-8b7920259261 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:24.507608 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0632c21d-8bc6-402e-8766-07f6ed3a2742 None None] MFA Rules not processed for user `f510a5006aa144bbaafb33feea40f83b`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:24.509968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0632c21d-8bc6-402e-8766-07f6ed3a2742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:24.659429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fb9b8-7aa4-407a-9eda-8b7920259261 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:24.674243 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f7e8e-bad0-45b6-863b-a1f1c34c3ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:24.675000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f7e8e-bad0-45b6-863b-a1f1c34c3ff5 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:24.675329 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f7e8e-bad0-45b6-863b-a1f1c34c3ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:24.675925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:24.676975 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f7e8e-bad0-45b6-863b-a1f1c34c3ff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:24.680093 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f7e8e-bad0-45b6-863b-a1f1c34c3ff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:24.684644 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f7e8e-bad0-45b6-863b-a1f1c34c3ff5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:24.684756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 811/1636] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:24 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:24.688946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e32508-0208-42d5-aec5-6baf94bf328e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:24.688946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e32508-0208-42d5-aec5-6baf94bf328e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:24.688946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e32508-0208-42d5-aec5-6baf94bf328e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:24.717850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 826/1637] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:24.717850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8551d-ebc1-482f-a44a-2cf34c686741 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:24.718567 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8551d-ebc1-482f-a44a-2cf34c686741 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:24.718567 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8551d-ebc1-482f-a44a-2cf34c686741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:24.750612 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7ba8551d-ebc1-482f-a44a-2cf34c686741 None None] MFA Rules not processed for user `b8751a4e92174d18b7b138392d2579a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:24.752537 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7ba8551d-ebc1-482f-a44a-2cf34c686741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:24.754338 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a3e32508-0208-42d5-aec5-6baf94bf328e None None] MFA Rules not processed for user `df74872b1cad46fca19b9e37035f38ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:24.755885 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a3e32508-0208-42d5-aec5-6baf94bf328e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:24.803863 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:24.909819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 827/1638] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 30 18:28:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:24.910796 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8551d-ebc1-482f-a44a-2cf34c686741 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:24.924955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 812/1639] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:24.924955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e32508-0208-42d5-aec5-6baf94bf328e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:24.949655 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e32508-0208-42d5-aec5-6baf94bf328e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:24.959834 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eaf75d-28f6-4e38-a3bc-50f2bd46d9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:24.960740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eaf75d-28f6-4e38-a3bc-50f2bd46d9c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:24.961021 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eaf75d-28f6-4e38-a3bc-50f2bd46d9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:24.961297 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eaf75d-28f6-4e38-a3bc-50f2bd46d9c7 admin admin] PATH_INFO: `/v3/projects/4a572f8d3641497f9f9be8866b5c89a1/users/659d03f3fe734f6985a6a84feba5b272/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:24.968370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eaf75d-28f6-4e38-a3bc-50f2bd46d9c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a572f8d3641497f9f9be8866b5c89a1, user_id=659d03f3fe734f6985a6a84feba5b272, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:24.973781 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eaf75d-28f6-4e38-a3bc-50f2bd46d9c7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:25.001049 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 813/1640] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:28:24 2022] PUT /identity/v3/projects/4a572f8d3641497f9f9be8866b5c89a1/users/659d03f3fe734f6985a6a84feba5b272/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:25.003258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eaf75d-28f6-4e38-a3bc-50f2bd46d9c7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:25.032482 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8551d-ebc1-482f-a44a-2cf34c686741 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:25.119862 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7253f9d-52cd-4119-acdc-028b30a2cea8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:25.120851 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7253f9d-52cd-4119-acdc-028b30a2cea8 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:25.121140 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7253f9d-52cd-4119-acdc-028b30a2cea8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:25.121419 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7253f9d-52cd-4119-acdc-028b30a2cea8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:25.130410 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eaf75d-28f6-4e38-a3bc-50f2bd46d9c7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:25.136731 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa5fe4-3c67-4413-8af5-69eb1a49d946 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:25.137753 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa5fe4-3c67-4413-8af5-69eb1a49d946 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:25.138575 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa5fe4-3c67-4413-8af5-69eb1a49d946 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:25.139151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa5fe4-3c67-4413-8af5-69eb1a49d946 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:25.237023 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7253f9d-52cd-4119-acdc-028b30a2cea8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:25.240017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7253f9d-52cd-4119-acdc-028b30a2cea8 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:25.258499 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 828/1641] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:25.260384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030f654-7dfb-4928-b038-3a533eb013bb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:25.260687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030f654-7dfb-4928-b038-3a533eb013bb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:25.260977 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030f654-7dfb-4928-b038-3a533eb013bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:25.289818 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa5fe4-3c67-4413-8af5-69eb1a49d946 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:25.291767 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa5fe4-3c67-4413-8af5-69eb1a49d946 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:25.311613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6030f654-7dfb-4928-b038-3a533eb013bb None None] MFA Rules not processed for user `659d03f3fe734f6985a6a84feba5b272`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:25.313114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6030f654-7dfb-4928-b038-3a533eb013bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:25.318930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 814/1642] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:25.366208 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:25.460170 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 829/1643] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:25 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:25.852922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa5fe4-3c67-4413-8af5-69eb1a49d946 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:25.864743 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa5fe4-3c67-4413-8af5-69eb1a49d946 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:25.877279 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9687caf-d5ad-49d1-b8a9-260001d6989a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:25.878430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9687caf-d5ad-49d1-b8a9-260001d6989a service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:25.883410 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9687caf-d5ad-49d1-b8a9-260001d6989a service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:25.883786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9687caf-d5ad-49d1-b8a9-260001d6989a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:25.914551 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:26.058567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9687caf-d5ad-49d1-b8a9-260001d6989a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:26.059718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9687caf-d5ad-49d1-b8a9-260001d6989a service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:26.079547 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 815/1644] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:25 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:26.221300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030f654-7dfb-4928-b038-3a533eb013bb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:26.237361 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030f654-7dfb-4928-b038-3a533eb013bb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:26.242994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd5c30-12d1-40cb-88ea-c03a766d96b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:26.244475 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd5c30-12d1-40cb-88ea-c03a766d96b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:26.245107 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd5c30-12d1-40cb-88ea-c03a766d96b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:26.245748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd5c30-12d1-40cb-88ea-c03a766d96b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:26.259807 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd5c30-12d1-40cb-88ea-c03a766d96b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:26.262469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd5c30-12d1-40cb-88ea-c03a766d96b5 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:26.284335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 830/1645] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:26 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:26.610766 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9687caf-d5ad-49d1-b8a9-260001d6989a service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:26.628300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd5c30-12d1-40cb-88ea-c03a766d96b5 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:26.635496 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9687caf-d5ad-49d1-b8a9-260001d6989a service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:26.640328 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd5c30-12d1-40cb-88ea-c03a766d96b5 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:26.644671 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1221f-bbb6-418a-a434-77d25dda9e10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:26.645931 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1221f-bbb6-418a-a434-77d25dda9e10 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:26.646904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64390fc-dcfe-4909-8b2c-0449425a6472 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:26.647434 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1221f-bbb6-418a-a434-77d25dda9e10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:26.647917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1221f-bbb6-418a-a434-77d25dda9e10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:26.648781 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64390fc-dcfe-4909-8b2c-0449425a6472 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:26.649861 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64390fc-dcfe-4909-8b2c-0449425a6472 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:26.650993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64390fc-dcfe-4909-8b2c-0449425a6472 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:26.756739 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64390fc-dcfe-4909-8b2c-0449425a6472 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:26.758640 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64390fc-dcfe-4909-8b2c-0449425a6472 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:26.788305 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 816/1646] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:26 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:26.795715 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1221f-bbb6-418a-a434-77d25dda9e10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:26.796548 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc97fc0-23b1-4437-8319-b35dd594840e None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:26.796936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc97fc0-23b1-4437-8319-b35dd594840e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:26.797235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc97fc0-23b1-4437-8319-b35dd594840e None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:26.797748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1221f-bbb6-418a-a434-77d25dda9e10 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:26.804824 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 817/1647] 10.222.0.33 () {54 vars in 961 bytes} [Thu Jun 30 18:28:26 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:28:26.812922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35ca2e-9560-43fe-a0e7-f26cda01da01 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:26.813442 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35ca2e-9560-43fe-a0e7-f26cda01da01 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:26.813755 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35ca2e-9560-43fe-a0e7-f26cda01da01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:26.816658 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 831/1648] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:26.851596 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0a35ca2e-9560-43fe-a0e7-f26cda01da01 None None] MFA Rules not processed for user `b8751a4e92174d18b7b138392d2579a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:26.853536 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0a35ca2e-9560-43fe-a0e7-f26cda01da01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:26.873882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01104bb1-afe3-469c-8ebd-bd103555a0ad None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:26.874296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01104bb1-afe3-469c-8ebd-bd103555a0ad None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:26.874796 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01104bb1-afe3-469c-8ebd-bd103555a0ad None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:26.876449 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 832/1649] 10.222.0.33 () {54 vars in 912 bytes} [Thu Jun 30 18:28:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:28:26.883837 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda527bb-7d9d-4128-9a99-6a1c26a8832e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:26.884239 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda527bb-7d9d-4128-9a99-6a1c26a8832e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:26.885185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda527bb-7d9d-4128-9a99-6a1c26a8832e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:26.888162 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 818/1650] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Jun 30 18:28:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:28:26.913711 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fda527bb-7d9d-4128-9a99-6a1c26a8832e None None] MFA Rules not processed for user `b8751a4e92174d18b7b138392d2579a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:26.915275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fda527bb-7d9d-4128-9a99-6a1c26a8832e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:26.944791 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 833/1651] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 30 18:28:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:28:27.136609 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35ca2e-9560-43fe-a0e7-f26cda01da01 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:27.151594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35ca2e-9560-43fe-a0e7-f26cda01da01 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:27.170134 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857fb176-914c-4c1c-ac9e-77078f47984b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:27.172104 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857fb176-914c-4c1c-ac9e-77078f47984b service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:27.172395 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857fb176-914c-4c1c-ac9e-77078f47984b service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:27.181817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857fb176-914c-4c1c-ac9e-77078f47984b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:27.217863 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857fb176-914c-4c1c-ac9e-77078f47984b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:27.222222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857fb176-914c-4c1c-ac9e-77078f47984b service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:27.272407 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 819/1652] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:31.209998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa53ec7-5a21-4a27-b8a6-ff00e1bd2314 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:31.210463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa53ec7-5a21-4a27-b8a6-ff00e1bd2314 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:31.211292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa53ec7-5a21-4a27-b8a6-ff00e1bd2314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:31.268761 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-dfa53ec7-5a21-4a27-b8a6-ff00e1bd2314 None None] MFA Rules not processed for user `c66efdb1cc27467d85831617590c8b28`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:31.270279 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-dfa53ec7-5a21-4a27-b8a6-ff00e1bd2314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:31.388045 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 834/1653] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:31.388589 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f070d1-b3bf-4c5b-9692-6348a10154e3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:31.389421 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f070d1-b3bf-4c5b-9692-6348a10154e3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:31.389739 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f070d1-b3bf-4c5b-9692-6348a10154e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:31.414069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-35f070d1-b3bf-4c5b-9692-6348a10154e3 None None] MFA Rules not processed for user `c66efdb1cc27467d85831617590c8b28`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:31.416159 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-35f070d1-b3bf-4c5b-9692-6348a10154e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:31.426484 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 820/1654] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:31.440837 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa53ec7-5a21-4a27-b8a6-ff00e1bd2314 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:31.456466 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa53ec7-5a21-4a27-b8a6-ff00e1bd2314 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:31.465088 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafbd1f8-3473-4469-bad3-3eef5d9a1a3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:31.466103 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbd1f8-3473-4469-bad3-3eef5d9a1a3f service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:31.466551 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbd1f8-3473-4469-bad3-3eef5d9a1a3f service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:31.466971 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbd1f8-3473-4469-bad3-3eef5d9a1a3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:31.478918 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafbd1f8-3473-4469-bad3-3eef5d9a1a3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:31.481768 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafbd1f8-3473-4469-bad3-3eef5d9a1a3f service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:31.499615 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 835/1655] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:31 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:32.300010 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50fb6b-d22d-4dd7-83d7-2368e79d950f None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:32.300300 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50fb6b-d22d-4dd7-83d7-2368e79d950f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:32.300581 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50fb6b-d22d-4dd7-83d7-2368e79d950f None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:32.302001 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 821/1656] 10.222.0.33 () {54 vars in 912 bytes} [Thu Jun 30 18:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:28:32.309894 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17117091-dae7-4e44-b009-843db2a4d1d7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:32.310972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17117091-dae7-4e44-b009-843db2a4d1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:32.311385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17117091-dae7-4e44-b009-843db2a4d1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:32.360030 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:32.378562 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-17117091-dae7-4e44-b009-843db2a4d1d7 None None] MFA Rules not processed for user `1d8c54102fb340859ed0acc3189fe376`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:32.383372 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-17117091-dae7-4e44-b009-843db2a4d1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:32.504849 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 836/1657] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 30 18:28:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:32.523216 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50fb6b-d22d-4dd7-83d7-2368e79d950f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:32.545870 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50fb6b-d22d-4dd7-83d7-2368e79d950f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:32.555881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a10382-f791-40a6-94f7-44214008bd9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:32.558020 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a10382-f791-40a6-94f7-44214008bd9b service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:32.558985 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a10382-f791-40a6-94f7-44214008bd9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:32.559548 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a10382-f791-40a6-94f7-44214008bd9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:32.574684 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a10382-f791-40a6-94f7-44214008bd9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:32.577331 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a10382-f791-40a6-94f7-44214008bd9b service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:32.636635 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 822/1658] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:33.328670 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a381b-0f36-4441-b5fc-02c2c2e86ba7 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:33.329176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a381b-0f36-4441-b5fc-02c2c2e86ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:33.329264 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a381b-0f36-4441-b5fc-02c2c2e86ba7 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:33.331102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 837/1659] 10.222.0.33 () {54 vars in 960 bytes} [Thu Jun 30 18:28:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:28:33.338728 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a76bc-2901-48f7-b0f7-fd810335bed0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:33.338943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a76bc-2901-48f7-b0f7-fd810335bed0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:33.339473 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a76bc-2901-48f7-b0f7-fd810335bed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:33.365463 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8a0a76bc-2901-48f7-b0f7-fd810335bed0 None None] MFA Rules not processed for user `49325c1733174e0c98b3475a10c9b5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:33.367139 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8a0a76bc-2901-48f7-b0f7-fd810335bed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:33.468855 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add25a3c-21da-4f35-ac41-6a8aba81b193 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:33.469147 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add25a3c-21da-4f35-ac41-6a8aba81b193 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:33.469761 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add25a3c-21da-4f35-ac41-6a8aba81b193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:33.504305 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:33.512920 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-add25a3c-21da-4f35-ac41-6a8aba81b193 None None] MFA Rules not processed for user `c66efdb1cc27467d85831617590c8b28`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:33.520422 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-add25a3c-21da-4f35-ac41-6a8aba81b193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:33.528783 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 823/1660] 10.222.0.33 () {58 vars in 1057 bytes} [Thu Jun 30 18:28:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:28:33.529948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a76bc-2901-48f7-b0f7-fd810335bed0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:33.534542 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 838/1661] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:28:33.547573 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a76bc-2901-48f7-b0f7-fd810335bed0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:33.555143 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b39d6-c9be-4c56-888b-8c8fc92e9c96 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49325c1733174e0c98b3475a10c9b5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:33.556092 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b39d6-c9be-4c56-888b-8c8fc92e9c96 service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:33.557394 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b39d6-c9be-4c56-888b-8c8fc92e9c96 service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:33.557927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b39d6-c9be-4c56-888b-8c8fc92e9c96 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:33.561037 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2b3b39d6-c9be-4c56-888b-8c8fc92e9c96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:33.583044 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b39d6-c9be-4c56-888b-8c8fc92e9c96 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:33.585043 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b39d6-c9be-4c56-888b-8c8fc92e9c96 service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:33.612951 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 824/1662] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:34.143385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee3f3e-1ea1-4696-9cc3-6ef732891af9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:34.144004 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee3f3e-1ea1-4696-9cc3-6ef732891af9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:34.144495 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee3f3e-1ea1-4696-9cc3-6ef732891af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:34.201673 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-65ee3f3e-1ea1-4696-9cc3-6ef732891af9 None None] MFA Rules not processed for user `db9762ad11e24246aefe2905eafe83f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:34.203233 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-65ee3f3e-1ea1-4696-9cc3-6ef732891af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:34.310047 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 839/1663] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:34.314044 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bc998-d9c7-4903-84b8-bfdd5b17ef48 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:34.315126 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bc998-d9c7-4903-84b8-bfdd5b17ef48 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:34.315595 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bc998-d9c7-4903-84b8-bfdd5b17ef48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:34.351767 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a67bc998-d9c7-4903-84b8-bfdd5b17ef48 None None] MFA Rules not processed for user `db9762ad11e24246aefe2905eafe83f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:34.353677 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a67bc998-d9c7-4903-84b8-bfdd5b17ef48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:34.368736 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 825/1664] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:34.381953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee3f3e-1ea1-4696-9cc3-6ef732891af9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:34.387180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5e4f6-41d3-40d6-a105-4b1e0bfb440d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:34.387563 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5e4f6-41d3-40d6-a105-4b1e0bfb440d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:34.387877 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5e4f6-41d3-40d6-a105-4b1e0bfb440d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:34.405533 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee3f3e-1ea1-4696-9cc3-6ef732891af9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:34.417974 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb60c1-424b-427f-b9ad-1dc3efc25ba9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:34.418817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb60c1-424b-427f-b9ad-1dc3efc25ba9 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:34.419191 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb60c1-424b-427f-b9ad-1dc3efc25ba9 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:34.419841 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb60c1-424b-427f-b9ad-1dc3efc25ba9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:34.433222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb60c1-424b-427f-b9ad-1dc3efc25ba9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:34.436054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb60c1-424b-427f-b9ad-1dc3efc25ba9 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:34.448800 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-35f5e4f6-41d3-40d6-a105-4b1e0bfb440d None None] MFA Rules not processed for user `f13e9e1bf5084d08919845767c70c3f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:34.450078 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-35f5e4f6-41d3-40d6-a105-4b1e0bfb440d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:34.454393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 840/1665] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:34.558341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 826/1666] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:34.733775 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb60c1-424b-427f-b9ad-1dc3efc25ba9 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:34.766436 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb60c1-424b-427f-b9ad-1dc3efc25ba9 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:34.783517 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ec75a-4241-41a3-b178-5992d00f1f28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:34.785559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ec75a-4241-41a3-b178-5992d00f1f28 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:34.786473 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ec75a-4241-41a3-b178-5992d00f1f28 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:34.787454 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ec75a-4241-41a3-b178-5992d00f1f28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:34.836143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ec75a-4241-41a3-b178-5992d00f1f28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:34.844448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ec75a-4241-41a3-b178-5992d00f1f28 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:34.875689 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 841/1667] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:35.111385 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9f507-9f14-4827-a60e-f0956d454614 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:35.113838 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9f507-9f14-4827-a60e-f0956d454614 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:35.114157 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9f507-9f14-4827-a60e-f0956d454614 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:35.116131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 827/1668] 10.222.0.33 () {54 vars in 912 bytes} [Thu Jun 30 18:28:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:28:35.122560 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac36ce0-c644-4664-b610-690758ad50ad None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:35.122954 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac36ce0-c644-4664-b610-690758ad50ad None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:35.123280 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac36ce0-c644-4664-b610-690758ad50ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:35.224181 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7ac36ce0-c644-4664-b610-690758ad50ad None None] MFA Rules not processed for user `1d8c54102fb340859ed0acc3189fe376`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:35.225443 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7ac36ce0-c644-4664-b610-690758ad50ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:35.246196 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 842/1669] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 30 18:28:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:28:35.258723 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b9f507-9f14-4827-a60e-f0956d454614 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:35.282073 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b9f507-9f14-4827-a60e-f0956d454614 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:35.293483 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb58c4-d13b-414d-a921-e663ae87e533 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:35.294352 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb58c4-d13b-414d-a921-e663ae87e533 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:35.294760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb58c4-d13b-414d-a921-e663ae87e533 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:35.295160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb58c4-d13b-414d-a921-e663ae87e533 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:35.333990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abb58c4-d13b-414d-a921-e663ae87e533 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:35.339026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abb58c4-d13b-414d-a921-e663ae87e533 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:35.366687 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 828/1670] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:36.527000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac36ce0-c644-4664-b610-690758ad50ad None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:36.539703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac36ce0-c644-4664-b610-690758ad50ad None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:36.553157 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8862dbcb-0969-4e44-ab08-93912759a52c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:36.553890 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862dbcb-0969-4e44-ab08-93912759a52c service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:36.554161 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862dbcb-0969-4e44-ab08-93912759a52c service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:36.555793 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862dbcb-0969-4e44-ab08-93912759a52c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:36.698234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8862dbcb-0969-4e44-ab08-93912759a52c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:36.699788 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8862dbcb-0969-4e44-ab08-93912759a52c service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:36.707547 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:36.724132 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 843/1671] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:36 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:38.891567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb384af-43da-4b6b-9076-b0b4d6e33583 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:38.893103 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb384af-43da-4b6b-9076-b0b4d6e33583 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:38.893827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb384af-43da-4b6b-9076-b0b4d6e33583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:38.923921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2bb384af-43da-4b6b-9076-b0b4d6e33583 None None] MFA Rules not processed for user `f13e9e1bf5084d08919845767c70c3f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:38.934307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2bb384af-43da-4b6b-9076-b0b4d6e33583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:38.959458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 829/1672] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:44.454754 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f048cb6-4ced-4da2-83d7-fa817d2a070e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:44.455217 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f048cb6-4ced-4da2-83d7-fa817d2a070e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:44.455315 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f048cb6-4ced-4da2-83d7-fa817d2a070e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:44.473883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1f048cb6-4ced-4da2-83d7-fa817d2a070e None None] MFA Rules not processed for user `1d8c54102fb340859ed0acc3189fe376`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:44.475092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1f048cb6-4ced-4da2-83d7-fa817d2a070e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:44.495721 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 844/1673] 10.222.0.18 () {58 vars in 1015 bytes} [Thu Jun 30 18:28:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:44.517653 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb384af-43da-4b6b-9076-b0b4d6e33583 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:44.526275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ddbac-4712-4cd7-9b0d-a005cedac4ff None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:44.526759 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ddbac-4712-4cd7-9b0d-a005cedac4ff None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:44.527191 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ddbac-4712-4cd7-9b0d-a005cedac4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:44.531033 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb384af-43da-4b6b-9076-b0b4d6e33583 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:44.541859 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc897ea-d253-4a4c-9f45-4e24c915eb0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:44.542865 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc897ea-d253-4a4c-9f45-4e24c915eb0a service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:44.543366 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc897ea-d253-4a4c-9f45-4e24c915eb0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:44.543785 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc897ea-d253-4a4c-9f45-4e24c915eb0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:44.552562 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7a4ddbac-4712-4cd7-9b0d-a005cedac4ff None None] MFA Rules not processed for user `4ddcf6306e704fe9ad5ed8779178ae67`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:44.554228 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7a4ddbac-4712-4cd7-9b0d-a005cedac4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:44.573660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc897ea-d253-4a4c-9f45-4e24c915eb0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:44.577710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc897ea-d253-4a4c-9f45-4e24c915eb0a service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:44.611118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 830/1674] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:44.676728 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 845/1675] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 30 18:28:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:28:44.843148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc897ea-d253-4a4c-9f45-4e24c915eb0a service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:44.844566 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ffc897ea-d253-4a4c-9f45-4e24c915eb0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:44.921348 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc897ea-d253-4a4c-9f45-4e24c915eb0a service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:44.931846 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6291918-fbf5-4334-9ee2-dd607300dd42 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bd521b9e93f4b3b8af64d46f36c019e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:44.932795 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6291918-fbf5-4334-9ee2-dd607300dd42 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:44.933566 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6291918-fbf5-4334-9ee2-dd607300dd42 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:44.934214 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6291918-fbf5-4334-9ee2-dd607300dd42 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:44.946735 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6291918-fbf5-4334-9ee2-dd607300dd42 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:44.948841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6291918-fbf5-4334-9ee2-dd607300dd42 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:44.981460 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 831/1676] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 30 18:28:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 18:28:47.088150 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c206357-28b9-4c6f-bd4f-9bb662a73c79 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:47.088647 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c206357-28b9-4c6f-bd4f-9bb662a73c79 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:47.088727 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c206357-28b9-4c6f-bd4f-9bb662a73c79 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:47.090448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 846/1677] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 30 18:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:28:47.137865 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517251ee-2ba8-48f4-9a13-10fe48539f8c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:47.138838 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517251ee-2ba8-48f4-9a13-10fe48539f8c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:47.138838 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517251ee-2ba8-48f4-9a13-10fe48539f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:47.168649 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-517251ee-2ba8-48f4-9a13-10fe48539f8c None None] MFA Rules not processed for user `b8751a4e92174d18b7b138392d2579a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:47.170550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-517251ee-2ba8-48f4-9a13-10fe48539f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:47.189910 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 832/1678] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 30 18:28:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:47.270205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c206357-28b9-4c6f-bd4f-9bb662a73c79 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:47.289404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c206357-28b9-4c6f-bd4f-9bb662a73c79 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:47.297701 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457eaf7-48ec-495a-ad87-543757ebbaf1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:47.298816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457eaf7-48ec-495a-ad87-543757ebbaf1 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:47.299345 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457eaf7-48ec-495a-ad87-543757ebbaf1 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:47.299769 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457eaf7-48ec-495a-ad87-543757ebbaf1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:47.317695 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457eaf7-48ec-495a-ad87-543757ebbaf1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:47.330071 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457eaf7-48ec-495a-ad87-543757ebbaf1 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:47.354431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 847/1679] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:47.656998 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5abee-bc9a-4bb5-998d-1c438f260532 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:47.657447 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5abee-bc9a-4bb5-998d-1c438f260532 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:47.657871 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5abee-bc9a-4bb5-998d-1c438f260532 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:47.660795 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 833/1680] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 30 18:28:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:28:47.668567 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80b448-95ec-4c51-a4d6-7e16a3baad91 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:47.669392 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80b448-95ec-4c51-a4d6-7e16a3baad91 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:47.669835 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80b448-95ec-4c51-a4d6-7e16a3baad91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:47.706047 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2b80b448-95ec-4c51-a4d6-7e16a3baad91 None None] MFA Rules not processed for user `b8751a4e92174d18b7b138392d2579a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:47.707375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2b80b448-95ec-4c51-a4d6-7e16a3baad91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:47.722945 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 848/1681] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 30 18:28:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:47.738888 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5abee-bc9a-4bb5-998d-1c438f260532 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:47.756274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5abee-bc9a-4bb5-998d-1c438f260532 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:47.768960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f618602-9cf0-4601-aad8-6f255a74b9a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:47.769705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f618602-9cf0-4601-aad8-6f255a74b9a2 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:47.770162 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f618602-9cf0-4601-aad8-6f255a74b9a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:47.770444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f618602-9cf0-4601-aad8-6f255a74b9a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:47.781218 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f618602-9cf0-4601-aad8-6f255a74b9a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:47.783102 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f618602-9cf0-4601-aad8-6f255a74b9a2 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:47.796217 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 834/1682] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:51.868704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80b448-95ec-4c51-a4d6-7e16a3baad91 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:51.941883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80b448-95ec-4c51-a4d6-7e16a3baad91 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:51.946907 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83c9f8-d72d-4649-aa61-dc3066226b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:51.947809 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83c9f8-d72d-4649-aa61-dc3066226b14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:51.948094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83c9f8-d72d-4649-aa61-dc3066226b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:51.948374 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83c9f8-d72d-4649-aa61-dc3066226b14 admin admin] PATH_INFO: `/v3/users/31234f42e4454706aa9295881d0abe0a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:51.949634 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83c9f8-d72d-4649-aa61-dc3066226b14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31234f42e4454706aa9295881d0abe0a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:51.951310 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: WARNING py.warnings [None req-5d83c9f8-d72d-4649-aa61-dc3066226b14 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 18:28:51.951310 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: warnings.warn(msg) Jun 30 18:28:51.951310 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: Jun 30 18:28:51.953707 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83c9f8-d72d-4649-aa61-dc3066226b14 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:52.046992 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5d83c9f8-d72d-4649-aa61-dc3066226b14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31234f42e4454706aa9295881d0abe0a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:52.094054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5d83c9f8-d72d-4649-aa61-dc3066226b14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31234f42e4454706aa9295881d0abe0a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:52.094938 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5d83c9f8-d72d-4649-aa61-dc3066226b14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31234f42e4454706aa9295881d0abe0a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:52.129716 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5d83c9f8-d72d-4649-aa61-dc3066226b14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31234f42e4454706aa9295881d0abe0a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:52.129716 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5d83c9f8-d72d-4649-aa61-dc3066226b14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31234f42e4454706aa9295881d0abe0a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:52.130919 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:52.154737 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 849/1683] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:28:51 2022] DELETE /identity/v3/users/31234f42e4454706aa9295881d0abe0a => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:52.159218 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd8d4f-5305-4211-9362-43fbfceb0480 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:52.159760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd8d4f-5305-4211-9362-43fbfceb0480 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:52.159866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd8d4f-5305-4211-9362-43fbfceb0480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:52.173327 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:52.179047 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-dddd8d4f-5305-4211-9362-43fbfceb0480 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:52.180330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dddd8d4f-5305-4211-9362-43fbfceb0480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:52.274106 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 835/1684] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:52.283567 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83c9f8-d72d-4649-aa61-dc3066226b14 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:52.285060 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5d83c9f8-d72d-4649-aa61-dc3066226b14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:52.456058 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83c9f8-d72d-4649-aa61-dc3066226b14 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:52.460566 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a521806-1cdf-4add-abf4-650799ae6f67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:52.462119 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a521806-1cdf-4add-abf4-650799ae6f67 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:52.463379 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a521806-1cdf-4add-abf4-650799ae6f67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:52.465455 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a521806-1cdf-4add-abf4-650799ae6f67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:52.483836 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a521806-1cdf-4add-abf4-650799ae6f67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:52.486567 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a521806-1cdf-4add-abf4-650799ae6f67 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:52.521870 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 850/1685] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:52.623161 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd8d4f-5305-4211-9362-43fbfceb0480 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:52.625181 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dddd8d4f-5305-4211-9362-43fbfceb0480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:52.642184 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd8d4f-5305-4211-9362-43fbfceb0480 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:52.650908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e630c76-aee9-41b9-8b12-ea0b73d39fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:52.652228 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e630c76-aee9-41b9-8b12-ea0b73d39fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:52.652849 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e630c76-aee9-41b9-8b12-ea0b73d39fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:52.653353 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e630c76-aee9-41b9-8b12-ea0b73d39fbc admin admin] PATH_INFO: `/v3/users/4f262bfdf0d54e76a77e8ce8cda137dc` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:52.655815 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e630c76-aee9-41b9-8b12-ea0b73d39fbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f262bfdf0d54e76a77e8ce8cda137dc)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:52.662136 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e630c76-aee9-41b9-8b12-ea0b73d39fbc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:52.726234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4e630c76-aee9-41b9-8b12-ea0b73d39fbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f262bfdf0d54e76a77e8ce8cda137dc'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:52.727369 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4e630c76-aee9-41b9-8b12-ea0b73d39fbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f262bfdf0d54e76a77e8ce8cda137dc'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:52.756286 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4e630c76-aee9-41b9-8b12-ea0b73d39fbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f262bfdf0d54e76a77e8ce8cda137dc'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:52.766483 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4e630c76-aee9-41b9-8b12-ea0b73d39fbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f262bfdf0d54e76a77e8ce8cda137dc'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:52.767117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4e630c76-aee9-41b9-8b12-ea0b73d39fbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f262bfdf0d54e76a77e8ce8cda137dc'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:52.786573 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 836/1686] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:28:52 2022] DELETE /identity/v3/users/4f262bfdf0d54e76a77e8ce8cda137dc => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:52.790935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84230bbf-5a15-43e8-b4d9-f4b9c49517c9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:52.791234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84230bbf-5a15-43e8-b4d9-f4b9c49517c9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:52.791549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84230bbf-5a15-43e8-b4d9-f4b9c49517c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:52.817973 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-84230bbf-5a15-43e8-b4d9-f4b9c49517c9 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:52.819205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-84230bbf-5a15-43e8-b4d9-f4b9c49517c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:52.929322 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 851/1687] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:52.949323 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e630c76-aee9-41b9-8b12-ea0b73d39fbc admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:52.951495 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4e630c76-aee9-41b9-8b12-ea0b73d39fbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:52.991248 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84230bbf-5a15-43e8-b4d9-f4b9c49517c9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:52.992926 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-84230bbf-5a15-43e8-b4d9-f4b9c49517c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:53.015811 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84230bbf-5a15-43e8-b4d9-f4b9c49517c9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:53.023858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9526b61-2ba6-4db5-9870-3f79448db3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:53.025038 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9526b61-2ba6-4db5-9870-3f79448db3a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:53.025514 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9526b61-2ba6-4db5-9870-3f79448db3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:53.026014 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9526b61-2ba6-4db5-9870-3f79448db3a9 admin admin] PATH_INFO: `/v3/projects/97d3ba408c3c4cc78d2a8e983869e11e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:53.031229 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9526b61-2ba6-4db5-9870-3f79448db3a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97d3ba408c3c4cc78d2a8e983869e11e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:53.033432 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9526b61-2ba6-4db5-9870-3f79448db3a9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:53.101839 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e630c76-aee9-41b9-8b12-ea0b73d39fbc admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:53.108946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8dc31f-9c03-4bf9-b72f-95bcf7372aab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:53.109869 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8dc31f-9c03-4bf9-b72f-95bcf7372aab service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:53.113392 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8dc31f-9c03-4bf9-b72f-95bcf7372aab service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:53.113711 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8dc31f-9c03-4bf9-b72f-95bcf7372aab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:53.145777 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8dc31f-9c03-4bf9-b72f-95bcf7372aab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:53.145777 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8dc31f-9c03-4bf9-b72f-95bcf7372aab service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:53.156800 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d9526b61-2ba6-4db5-9870-3f79448db3a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97d3ba408c3c4cc78d2a8e983869e11e', 'request_id': 'req-d9526b61-2ba6-4db5-9870-3f79448db3a9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:53.171757 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 837/1688] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:53.180628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d9526b61-2ba6-4db5-9870-3f79448db3a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97d3ba408c3c4cc78d2a8e983869e11e', 'request_id': 'req-d9526b61-2ba6-4db5-9870-3f79448db3a9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:53.191157 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d9526b61-2ba6-4db5-9870-3f79448db3a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97d3ba408c3c4cc78d2a8e983869e11e', 'request_id': 'req-d9526b61-2ba6-4db5-9870-3f79448db3a9'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:53.226831 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d9526b61-2ba6-4db5-9870-3f79448db3a9 admin admin] The token cache is being invalidate because project 97d3ba408c3c4cc78d2a8e983869e11e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:28:53.227116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d9526b61-2ba6-4db5-9870-3f79448db3a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:53.228012 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d9526b61-2ba6-4db5-9870-3f79448db3a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:53.230040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 852/1689] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:28:52 2022] DELETE /identity/v3/projects/97d3ba408c3c4cc78d2a8e983869e11e => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:53.235347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8dc31f-9c03-4bf9-b72f-95bcf7372aab service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:53.260288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8dc31f-9c03-4bf9-b72f-95bcf7372aab service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:53.303496 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:53.414358 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91bd169-3e4f-47ec-b8be-9efca7fac194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:53.415641 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bd169-3e4f-47ec-b8be-9efca7fac194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:53.417316 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bd169-3e4f-47ec-b8be-9efca7fac194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:53.417909 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bd169-3e4f-47ec-b8be-9efca7fac194 admin admin] PATH_INFO: `/v3/users/df74872b1cad46fca19b9e37035f38ef` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:53.422489 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91bd169-3e4f-47ec-b8be-9efca7fac194 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df74872b1cad46fca19b9e37035f38ef)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:53.428810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91bd169-3e4f-47ec-b8be-9efca7fac194 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:53.598838 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f91bd169-3e4f-47ec-b8be-9efca7fac194 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df74872b1cad46fca19b9e37035f38ef'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:53.600039 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f91bd169-3e4f-47ec-b8be-9efca7fac194 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df74872b1cad46fca19b9e37035f38ef'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:53.617702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f91bd169-3e4f-47ec-b8be-9efca7fac194 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df74872b1cad46fca19b9e37035f38ef'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:53.630394 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f91bd169-3e4f-47ec-b8be-9efca7fac194 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df74872b1cad46fca19b9e37035f38ef'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:53.631307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f91bd169-3e4f-47ec-b8be-9efca7fac194 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df74872b1cad46fca19b9e37035f38ef'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:53.650110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 838/1690] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:28:53 2022] DELETE /identity/v3/users/df74872b1cad46fca19b9e37035f38ef => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:53.821911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9526b61-2ba6-4db5-9870-3f79448db3a9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:53.823549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d9526b61-2ba6-4db5-9870-3f79448db3a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:53.871257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9526b61-2ba6-4db5-9870-3f79448db3a9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:53.907915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:54.014103 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0a9ac-d0f9-4f0e-b965-df70e2255a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:54.016000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0a9ac-d0f9-4f0e-b965-df70e2255a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:54.016000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0a9ac-d0f9-4f0e-b965-df70e2255a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:54.016183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0a9ac-d0f9-4f0e-b965-df70e2255a75 admin admin] PATH_INFO: `/v3/projects/6a127ed10b0a465aa487242a344defa0` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:54.017804 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d0a9ac-d0f9-4f0e-b965-df70e2255a75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a127ed10b0a465aa487242a344defa0)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:54.020087 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d0a9ac-d0f9-4f0e-b965-df70e2255a75 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:54.119065 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-69d0a9ac-d0f9-4f0e-b965-df70e2255a75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a127ed10b0a465aa487242a344defa0', 'request_id': 'req-69d0a9ac-d0f9-4f0e-b965-df70e2255a75'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:54.126671 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-69d0a9ac-d0f9-4f0e-b965-df70e2255a75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a127ed10b0a465aa487242a344defa0', 'request_id': 'req-69d0a9ac-d0f9-4f0e-b965-df70e2255a75'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:54.133782 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-69d0a9ac-d0f9-4f0e-b965-df70e2255a75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a127ed10b0a465aa487242a344defa0', 'request_id': 'req-69d0a9ac-d0f9-4f0e-b965-df70e2255a75'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:54.151137 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-69d0a9ac-d0f9-4f0e-b965-df70e2255a75 admin admin] The token cache is being invalidate because project 6a127ed10b0a465aa487242a344defa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:28:54.151533 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-69d0a9ac-d0f9-4f0e-b965-df70e2255a75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:54.152620 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-69d0a9ac-d0f9-4f0e-b965-df70e2255a75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:54.158394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 853/1691] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:28:53 2022] DELETE /identity/v3/projects/6a127ed10b0a465aa487242a344defa0 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:54.160659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47c7f0-f37e-4c4c-81df-2ab5a08a7f78 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:54.160982 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47c7f0-f37e-4c4c-81df-2ab5a08a7f78 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:54.161266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47c7f0-f37e-4c4c-81df-2ab5a08a7f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:54.183480 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-cb47c7f0-f37e-4c4c-81df-2ab5a08a7f78 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:54.184725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cb47c7f0-f37e-4c4c-81df-2ab5a08a7f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:54.297532 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 839/1692] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:54.310909 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2a068-d3c0-4a34-aea6-d14623cce7c4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:54.311495 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2a068-d3c0-4a34-aea6-d14623cce7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:54.311772 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2a068-d3c0-4a34-aea6-d14623cce7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:54.351851 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-a1f2a068-d3c0-4a34-aea6-d14623cce7c4 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:54.353852 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a1f2a068-d3c0-4a34-aea6-d14623cce7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:54.368251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 854/1693] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:54.369552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47c7f0-f37e-4c4c-81df-2ab5a08a7f78 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:54.383052 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47c7f0-f37e-4c4c-81df-2ab5a08a7f78 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:54.393483 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f71a900-54db-4c15-a082-e200b57419e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:54.398357 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71a900-54db-4c15-a082-e200b57419e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:54.398943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71a900-54db-4c15-a082-e200b57419e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:54.399404 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71a900-54db-4c15-a082-e200b57419e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:54.400428 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f71a900-54db-4c15-a082-e200b57419e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:28:54.401071 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f71a900-54db-4c15-a082-e200b57419e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:54.409833 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f71a900-54db-4c15-a082-e200b57419e8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:54.435807 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 840/1694] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:28:54.437038 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f71a900-54db-4c15-a082-e200b57419e8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:54.437398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2a068-d3c0-4a34-aea6-d14623cce7c4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:54.442740 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a1f2a068-d3c0-4a34-aea6-d14623cce7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:54.459040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2a068-d3c0-4a34-aea6-d14623cce7c4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:54.464022 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede567fa-0e25-4f66-a83b-01d194add786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:54.464981 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede567fa-0e25-4f66-a83b-01d194add786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:54.465342 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede567fa-0e25-4f66-a83b-01d194add786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:54.465753 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede567fa-0e25-4f66-a83b-01d194add786 admin admin] PATH_INFO: `/v3/projects/ecbee9bc0f7749b386ad738063690e0c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:54.467849 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede567fa-0e25-4f66-a83b-01d194add786 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecbee9bc0f7749b386ad738063690e0c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:54.470590 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede567fa-0e25-4f66-a83b-01d194add786 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:54.495888 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f71a900-54db-4c15-a082-e200b57419e8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:54.514870 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf638b6-70ec-490f-8185-719b96d46f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:54.515785 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf638b6-70ec-490f-8185-719b96d46f89 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:54.516219 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf638b6-70ec-490f-8185-719b96d46f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:54.516646 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf638b6-70ec-490f-8185-719b96d46f89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:54.517854 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf638b6-70ec-490f-8185-719b96d46f89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:54.520131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf638b6-70ec-490f-8185-719b96d46f89 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:54.559267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ede567fa-0e25-4f66-a83b-01d194add786 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecbee9bc0f7749b386ad738063690e0c', 'request_id': 'req-ede567fa-0e25-4f66-a83b-01d194add786'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:54.568455 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ede567fa-0e25-4f66-a83b-01d194add786 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecbee9bc0f7749b386ad738063690e0c', 'request_id': 'req-ede567fa-0e25-4f66-a83b-01d194add786'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:54.577677 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ede567fa-0e25-4f66-a83b-01d194add786 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecbee9bc0f7749b386ad738063690e0c', 'request_id': 'req-ede567fa-0e25-4f66-a83b-01d194add786'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:54.587524 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 841/1695] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:28:54 2022] POST /identity/v3/projects => generated 371 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:54.588783 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf638b6-70ec-490f-8185-719b96d46f89 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:54.608428 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ede567fa-0e25-4f66-a83b-01d194add786 admin admin] The token cache is being invalidate because project ecbee9bc0f7749b386ad738063690e0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:28:54.608732 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ede567fa-0e25-4f66-a83b-01d194add786 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:54.609490 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ede567fa-0e25-4f66-a83b-01d194add786 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:54.615394 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:54.617859 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 855/1696] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:28:54 2022] DELETE /identity/v3/projects/ecbee9bc0f7749b386ad738063690e0c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:28:54.619459 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670ba09-ff0a-47cd-923f-a3a2724705f8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:54.619811 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670ba09-ff0a-47cd-923f-a3a2724705f8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:54.620052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670ba09-ff0a-47cd-923f-a3a2724705f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:54.639935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d670ba09-ff0a-47cd-923f-a3a2724705f8 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:54.641203 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d670ba09-ff0a-47cd-923f-a3a2724705f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:54.679927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:54.698150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf638b6-70ec-490f-8185-719b96d46f89 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:54.704443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e4db4-5b34-45aa-bb41-1f062f7fa45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:54.705127 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e4db4-5b34-45aa-bb41-1f062f7fa45d admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:54.705419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e4db4-5b34-45aa-bb41-1f062f7fa45d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:54.705732 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e4db4-5b34-45aa-bb41-1f062f7fa45d admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:54.706445 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e4db4-5b34-45aa-bb41-1f062f7fa45d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:54.708410 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e4db4-5b34-45aa-bb41-1f062f7fa45d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:54.735876 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 856/1697] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:54.740278 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 842/1698] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:28:54 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:54.744249 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670ba09-ff0a-47cd-923f-a3a2724705f8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:54.744730 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09169e-a621-4790-9e33-75b1ae539946 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:54.745030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09169e-a621-4790-9e33-75b1ae539946 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:54.745321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09169e-a621-4790-9e33-75b1ae539946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:54.745925 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d670ba09-ff0a-47cd-923f-a3a2724705f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:54.760158 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670ba09-ff0a-47cd-923f-a3a2724705f8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:54.763187 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ff09169e-a621-4790-9e33-75b1ae539946 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:54.764999 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ff09169e-a621-4790-9e33-75b1ae539946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:54.765757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc043bd-cf2a-455b-b2bd-6ca5b2b9b307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:54.766418 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc043bd-cf2a-455b-b2bd-6ca5b2b9b307 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:54.766703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc043bd-cf2a-455b-b2bd-6ca5b2b9b307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:54.766981 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc043bd-cf2a-455b-b2bd-6ca5b2b9b307 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:54.767710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc043bd-cf2a-455b-b2bd-6ca5b2b9b307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:54.769384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc043bd-cf2a-455b-b2bd-6ca5b2b9b307 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:54.773318 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 843/1699] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:28:54.777318 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09169e-a621-4790-9e33-75b1ae539946 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:54.781121 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 857/1700] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:28:54.785007 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc043bd-cf2a-455b-b2bd-6ca5b2b9b307 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:54.793141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09169e-a621-4790-9e33-75b1ae539946 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:54.795330 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc043bd-cf2a-455b-b2bd-6ca5b2b9b307 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:54.798144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a32626-bf6a-46d8-a383-d612d4c0ffcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:54.798821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a32626-bf6a-46d8-a383-d612d4c0ffcc admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:54.799212 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a32626-bf6a-46d8-a383-d612d4c0ffcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:54.799495 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a32626-bf6a-46d8-a383-d612d4c0ffcc admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:54.800247 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a32626-bf6a-46d8-a383-d612d4c0ffcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:28:54.800355 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333aabae-9b29-4d95-89c7-deec4c96827d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:54.800697 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a32626-bf6a-46d8-a383-d612d4c0ffcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:54.800927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333aabae-9b29-4d95-89c7-deec4c96827d admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:54.801212 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333aabae-9b29-4d95-89c7-deec4c96827d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:54.801514 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333aabae-9b29-4d95-89c7-deec4c96827d admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:54.802269 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333aabae-9b29-4d95-89c7-deec4c96827d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:54.802436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a32626-bf6a-46d8-a383-d612d4c0ffcc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:54.803973 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333aabae-9b29-4d95-89c7-deec4c96827d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:54.818168 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 858/1701] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:28:54.818344 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 844/1702] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:28:54.819049 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a32626-bf6a-46d8-a383-d612d4c0ffcc admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:54.820215 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333aabae-9b29-4d95-89c7-deec4c96827d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:54.829292 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a32626-bf6a-46d8-a383-d612d4c0ffcc admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:54.830629 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333aabae-9b29-4d95-89c7-deec4c96827d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:54.833980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e594f-5a16-42c9-9c00-f817e0e43d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:54.834612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e594f-5a16-42c9-9c00-f817e0e43d6b admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:54.834897 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e594f-5a16-42c9-9c00-f817e0e43d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:54.835193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e594f-5a16-42c9-9c00-f817e0e43d6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:54.835298 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de968bb-e527-48f9-ab6b-9114ed3b381c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:54.835865 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e594f-5a16-42c9-9c00-f817e0e43d6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:54.836045 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de968bb-e527-48f9-ab6b-9114ed3b381c admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:54.836327 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de968bb-e527-48f9-ab6b-9114ed3b381c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:54.836625 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de968bb-e527-48f9-ab6b-9114ed3b381c admin admin] PATH_INFO: `/v3/projects/c83e905f79a242d6897cb0bbb4e94712/users/46f996ff9e124483ad8dcc0163299159/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:54.837799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e594f-5a16-42c9-9c00-f817e0e43d6b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:54.853090 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de968bb-e527-48f9-ab6b-9114ed3b381c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c83e905f79a242d6897cb0bbb4e94712, user_id=46f996ff9e124483ad8dcc0163299159, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:54.855824 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de968bb-e527-48f9-ab6b-9114ed3b381c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:54.870331 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 859/1703] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:28:54 2022] PUT /identity/v3/projects/c83e905f79a242d6897cb0bbb4e94712/users/46f996ff9e124483ad8dcc0163299159/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:28:54.870708 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 845/1704] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:28:54 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:54.878791 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de968bb-e527-48f9-ab6b-9114ed3b381c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:54.882426 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2739d-e149-41c6-a3e2-3135c58fa707 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:54.882729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2739d-e149-41c6-a3e2-3135c58fa707 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:54.883014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2739d-e149-41c6-a3e2-3135c58fa707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:54.915736 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c2c2739d-e149-41c6-a3e2-3135c58fa707 None None] MFA Rules not processed for user `46f996ff9e124483ad8dcc0163299159`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:54.917144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c2c2739d-e149-41c6-a3e2-3135c58fa707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:54.947392 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de968bb-e527-48f9-ab6b-9114ed3b381c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:55.019318 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 846/1705] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:55.024223 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc39f9-f7f4-4256-9bce-dff2330efcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:55.025006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc39f9-f7f4-4256-9bce-dff2330efcd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:55.025187 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc39f9-f7f4-4256-9bce-dff2330efcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:55.025463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc39f9-f7f4-4256-9bce-dff2330efcd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:55.026198 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efc39f9-f7f4-4256-9bce-dff2330efcd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:55.028356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efc39f9-f7f4-4256-9bce-dff2330efcd6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:55.052293 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 860/1706] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:28:54 2022] POST /identity/v3/users => generated 287 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:55.053532 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2739d-e149-41c6-a3e2-3135c58fa707 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:55.070121 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2739d-e149-41c6-a3e2-3135c58fa707 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:55.074740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9430e1-a99f-4522-8e2e-5af64cf3d747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:55.075402 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9430e1-a99f-4522-8e2e-5af64cf3d747 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:55.075688 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9430e1-a99f-4522-8e2e-5af64cf3d747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:55.076542 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9430e1-a99f-4522-8e2e-5af64cf3d747 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:55.077364 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9430e1-a99f-4522-8e2e-5af64cf3d747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:55.079129 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9430e1-a99f-4522-8e2e-5af64cf3d747 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:55.090261 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 847/1707] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:55.094272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc39f9-f7f4-4256-9bce-dff2330efcd6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:55.111303 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc39f9-f7f4-4256-9bce-dff2330efcd6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:55.116830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08371e7-91ee-499e-b7ac-76fef7de1c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:55.117989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08371e7-91ee-499e-b7ac-76fef7de1c96 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:55.118465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08371e7-91ee-499e-b7ac-76fef7de1c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:55.119015 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08371e7-91ee-499e-b7ac-76fef7de1c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:55.119936 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08371e7-91ee-499e-b7ac-76fef7de1c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:55.121798 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08371e7-91ee-499e-b7ac-76fef7de1c96 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:55.135174 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 861/1708] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:28:55.139992 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9430e1-a99f-4522-8e2e-5af64cf3d747 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:55.149031 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08371e7-91ee-499e-b7ac-76fef7de1c96 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:55.150771 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b08371e7-91ee-499e-b7ac-76fef7de1c96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:55.151542 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9430e1-a99f-4522-8e2e-5af64cf3d747 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:55.156832 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a0fa0-f619-4efc-a79f-83f38e9e4d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:55.157739 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a0fa0-f619-4efc-a79f-83f38e9e4d37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:55.158021 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a0fa0-f619-4efc-a79f-83f38e9e4d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:55.158297 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a0fa0-f619-4efc-a79f-83f38e9e4d37 admin admin] PATH_INFO: `/v3/projects/504cb727d0af40e6b5e95a8f85e39cc3/users/44c75ba167424fe99063189343ab9561/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:55.166026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:55.175489 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a0fa0-f619-4efc-a79f-83f38e9e4d37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=504cb727d0af40e6b5e95a8f85e39cc3, user_id=44c75ba167424fe99063189343ab9561, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:55.178374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a0fa0-f619-4efc-a79f-83f38e9e4d37 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:55.194269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 848/1709] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:28:55 2022] PUT /identity/v3/projects/504cb727d0af40e6b5e95a8f85e39cc3/users/44c75ba167424fe99063189343ab9561/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:55.204855 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d59de-9daa-48bb-ab54-7b9c7257c046 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:55.205146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d59de-9daa-48bb-ab54-7b9c7257c046 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:55.205430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d59de-9daa-48bb-ab54-7b9c7257c046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:55.227664 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:55.241115 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-090d59de-9daa-48bb-ab54-7b9c7257c046 None None] MFA Rules not processed for user `44c75ba167424fe99063189343ab9561`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:55.242450 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-090d59de-9daa-48bb-ab54-7b9c7257c046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:55.242559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08371e7-91ee-499e-b7ac-76fef7de1c96 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:55.313281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 849/1710] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:28:55.323101 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c213d4-8db7-4c14-8755-0abd0307526b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:55.323770 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c213d4-8db7-4c14-8755-0abd0307526b service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:55.324065 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c213d4-8db7-4c14-8755-0abd0307526b service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:55.324411 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c213d4-8db7-4c14-8755-0abd0307526b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:55.325482 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d59de-9daa-48bb-ab54-7b9c7257c046 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:55.327718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-090d59de-9daa-48bb-ab54-7b9c7257c046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:55.385911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d59de-9daa-48bb-ab54-7b9c7257c046 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:55.390145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78cc5b0-a0c8-4118-bfc1-a7bec3491a8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:55.390794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cc5b0-a0c8-4118-bfc1-a7bec3491a8e service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:55.391106 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cc5b0-a0c8-4118-bfc1-a7bec3491a8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:55.391410 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cc5b0-a0c8-4118-bfc1-a7bec3491a8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:55.395018 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c213d4-8db7-4c14-8755-0abd0307526b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:55.396633 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c213d4-8db7-4c14-8755-0abd0307526b service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:55.399793 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78cc5b0-a0c8-4118-bfc1-a7bec3491a8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:55.401225 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78cc5b0-a0c8-4118-bfc1-a7bec3491a8e service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:55.406768 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 862/1711] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:28:55.410777 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 850/1712] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:28:55.432261 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c213d4-8db7-4c14-8755-0abd0307526b service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:55.487774 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:55.507605 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c213d4-8db7-4c14-8755-0abd0307526b service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:55.514920 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e7bb3-ab95-459e-b867-5f7e877f744a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:55.516019 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e7bb3-ab95-459e-b867-5f7e877f744a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:55.516417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e7bb3-ab95-459e-b867-5f7e877f744a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:55.516774 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e7bb3-ab95-459e-b867-5f7e877f744a admin admin] PATH_INFO: `/v3/users/44c75ba167424fe99063189343ab9561` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:55.518105 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e7bb3-ab95-459e-b867-5f7e877f744a admin admin] RBAC: Authorizing `identity:delete_user(user_id=44c75ba167424fe99063189343ab9561)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:55.520345 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e7bb3-ab95-459e-b867-5f7e877f744a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:55.577510 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c72e7bb3-ab95-459e-b867-5f7e877f744a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44c75ba167424fe99063189343ab9561'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:55.593455 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c72e7bb3-ab95-459e-b867-5f7e877f744a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44c75ba167424fe99063189343ab9561'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:55.593666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c72e7bb3-ab95-459e-b867-5f7e877f744a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44c75ba167424fe99063189343ab9561'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:55.602262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c72e7bb3-ab95-459e-b867-5f7e877f744a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44c75ba167424fe99063189343ab9561'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:55.603361 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c72e7bb3-ab95-459e-b867-5f7e877f744a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44c75ba167424fe99063189343ab9561'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:55.622717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 863/1713] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:28:55 2022] DELETE /identity/v3/users/44c75ba167424fe99063189343ab9561 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:55.631439 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63557ad5-f5b8-4e80-b660-4d1ccb00829a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:55.631764 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63557ad5-f5b8-4e80-b660-4d1ccb00829a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:55.632089 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63557ad5-f5b8-4e80-b660-4d1ccb00829a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:55.652493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-63557ad5-f5b8-4e80-b660-4d1ccb00829a None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:55.653744 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-63557ad5-f5b8-4e80-b660-4d1ccb00829a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:55.720086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 851/1714] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:55.730303 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e7bb3-ab95-459e-b867-5f7e877f744a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:55.731980 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c72e7bb3-ab95-459e-b867-5f7e877f744a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:55.806243 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e7bb3-ab95-459e-b867-5f7e877f744a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:55.810513 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc49ce-5f49-48f3-9f25-57efafcc8218 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:55.811551 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc49ce-5f49-48f3-9f25-57efafcc8218 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:55.811893 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc49ce-5f49-48f3-9f25-57efafcc8218 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:55.812189 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc49ce-5f49-48f3-9f25-57efafcc8218 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:55.822309 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc49ce-5f49-48f3-9f25-57efafcc8218 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:55.825226 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc49ce-5f49-48f3-9f25-57efafcc8218 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:55.837093 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 864/1715] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:56.126647 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63557ad5-f5b8-4e80-b660-4d1ccb00829a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:56.128279 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-63557ad5-f5b8-4e80-b660-4d1ccb00829a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:56.147594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63557ad5-f5b8-4e80-b660-4d1ccb00829a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:56.183821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77627dd-9927-4f9f-9243-68cd9a3c2d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:56.184721 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77627dd-9927-4f9f-9243-68cd9a3c2d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:56.185002 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77627dd-9927-4f9f-9243-68cd9a3c2d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:56.186662 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77627dd-9927-4f9f-9243-68cd9a3c2d25 admin admin] PATH_INFO: `/v3/projects/504cb727d0af40e6b5e95a8f85e39cc3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:56.188131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77627dd-9927-4f9f-9243-68cd9a3c2d25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=504cb727d0af40e6b5e95a8f85e39cc3)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:56.189178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: WARNING py.warnings [None req-d77627dd-9927-4f9f-9243-68cd9a3c2d25 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 18:28:56.189178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: warnings.warn(msg) Jun 30 18:28:56.189178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: Jun 30 18:28:56.191275 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77627dd-9927-4f9f-9243-68cd9a3c2d25 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:56.244392 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:56.261796 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d77627dd-9927-4f9f-9243-68cd9a3c2d25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '504cb727d0af40e6b5e95a8f85e39cc3', 'request_id': 'req-d77627dd-9927-4f9f-9243-68cd9a3c2d25'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:56.280540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d77627dd-9927-4f9f-9243-68cd9a3c2d25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '504cb727d0af40e6b5e95a8f85e39cc3', 'request_id': 'req-d77627dd-9927-4f9f-9243-68cd9a3c2d25'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:56.289386 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d77627dd-9927-4f9f-9243-68cd9a3c2d25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '504cb727d0af40e6b5e95a8f85e39cc3', 'request_id': 'req-d77627dd-9927-4f9f-9243-68cd9a3c2d25'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:56.309495 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d77627dd-9927-4f9f-9243-68cd9a3c2d25 admin admin] The token cache is being invalidate because project 504cb727d0af40e6b5e95a8f85e39cc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:28:56.309851 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d77627dd-9927-4f9f-9243-68cd9a3c2d25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:56.314103 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d77627dd-9927-4f9f-9243-68cd9a3c2d25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:56.316177 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 852/1716] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:28:56 2022] DELETE /identity/v3/projects/504cb727d0af40e6b5e95a8f85e39cc3 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:56.320853 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30313e-33ee-4762-a817-b733d0fdd818 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:56.321369 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30313e-33ee-4762-a817-b733d0fdd818 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:56.321455 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30313e-33ee-4762-a817-b733d0fdd818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:56.355463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0d30313e-33ee-4762-a817-b733d0fdd818 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:56.356868 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0d30313e-33ee-4762-a817-b733d0fdd818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:56.429278 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 865/1717] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:56.432169 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315d7f1-cbaf-4950-9ae8-8f68c2f1603f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:56.432466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315d7f1-cbaf-4950-9ae8-8f68c2f1603f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:56.432755 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315d7f1-cbaf-4950-9ae8-8f68c2f1603f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:56.469405 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3315d7f1-cbaf-4950-9ae8-8f68c2f1603f None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:56.483431 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3315d7f1-cbaf-4950-9ae8-8f68c2f1603f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:56.498993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 853/1718] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:56.503100 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d30313e-33ee-4762-a817-b733d0fdd818 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:56.516675 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d30313e-33ee-4762-a817-b733d0fdd818 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:56.534229 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65c394-fc0f-4c78-abf7-2d7ce4be0b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:56.535217 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65c394-fc0f-4c78-abf7-2d7ce4be0b99 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:56.535709 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65c394-fc0f-4c78-abf7-2d7ce4be0b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:56.536141 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65c394-fc0f-4c78-abf7-2d7ce4be0b99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:56.536744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65c394-fc0f-4c78-abf7-2d7ce4be0b99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:28:56.537099 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65c394-fc0f-4c78-abf7-2d7ce4be0b99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:56.539128 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65c394-fc0f-4c78-abf7-2d7ce4be0b99 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:56.551997 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:56.572353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 866/1719] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:28:56.572632 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3315d7f1-cbaf-4950-9ae8-8f68c2f1603f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:56.592349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3315d7f1-cbaf-4950-9ae8-8f68c2f1603f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:56.614347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d20409-a6b4-4c17-8e29-0ac939a6e36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:56.615161 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d20409-a6b4-4c17-8e29-0ac939a6e36e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:56.615548 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d20409-a6b4-4c17-8e29-0ac939a6e36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:56.615989 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d20409-a6b4-4c17-8e29-0ac939a6e36e admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:56.616756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d20409-a6b4-4c17-8e29-0ac939a6e36e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:56.618825 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d20409-a6b4-4c17-8e29-0ac939a6e36e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:56.652544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 854/1720] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:28:56 2022] POST /identity/v3/projects => generated 377 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:56.658018 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65c394-fc0f-4c78-abf7-2d7ce4be0b99 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:56.675257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65c394-fc0f-4c78-abf7-2d7ce4be0b99 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:56.759270 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7cff0-8f11-4701-b1ca-fade0f2348c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:56.759951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7cff0-8f11-4701-b1ca-fade0f2348c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:56.760240 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7cff0-8f11-4701-b1ca-fade0f2348c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:56.760522 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7cff0-8f11-4701-b1ca-fade0f2348c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:56.761231 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7cff0-8f11-4701-b1ca-fade0f2348c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:56.763897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7cff0-8f11-4701-b1ca-fade0f2348c7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:56.785192 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 867/1721] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:28:56 2022] POST /identity/v3/users => generated 289 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:28:56.789463 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d20409-a6b4-4c17-8e29-0ac939a6e36e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:56.801465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d20409-a6b4-4c17-8e29-0ac939a6e36e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:56.806548 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791eb210-1614-49b1-876d-b2bada534433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:56.807226 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791eb210-1614-49b1-876d-b2bada534433 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:56.807512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791eb210-1614-49b1-876d-b2bada534433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:56.807788 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791eb210-1614-49b1-876d-b2bada534433 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:56.808554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791eb210-1614-49b1-876d-b2bada534433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:56.810838 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791eb210-1614-49b1-876d-b2bada534433 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:56.826536 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 855/1722] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:56.828636 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7cff0-8f11-4701-b1ca-fade0f2348c7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:56.839401 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7cff0-8f11-4701-b1ca-fade0f2348c7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:56.844945 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6c9e4-3ff8-41ee-bad0-e0c8186b517c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:56.845921 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6c9e4-3ff8-41ee-bad0-e0c8186b517c admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:56.850594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6c9e4-3ff8-41ee-bad0-e0c8186b517c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:56.851146 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6c9e4-3ff8-41ee-bad0-e0c8186b517c admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:56.851866 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6c9e4-3ff8-41ee-bad0-e0c8186b517c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:56.853527 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6c9e4-3ff8-41ee-bad0-e0c8186b517c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:56.868819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 868/1723] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:56.873603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791eb210-1614-49b1-876d-b2bada534433 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:56.883583 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791eb210-1614-49b1-876d-b2bada534433 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:56.888395 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1521ea-fbbe-4559-9ad3-2b6792332049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:56.889319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1521ea-fbbe-4559-9ad3-2b6792332049 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:56.889549 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1521ea-fbbe-4559-9ad3-2b6792332049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:56.889861 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1521ea-fbbe-4559-9ad3-2b6792332049 admin admin] PATH_INFO: `/v3/projects/0b32a883f9ee4186b559190a6cddf33f/users/c6edb603589b4b1984b3b2a9df6c8151/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:56.904535 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1521ea-fbbe-4559-9ad3-2b6792332049 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b32a883f9ee4186b559190a6cddf33f, user_id=c6edb603589b4b1984b3b2a9df6c8151, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:56.907271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1521ea-fbbe-4559-9ad3-2b6792332049 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:56.926132 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 856/1724] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:28:56 2022] PUT /identity/v3/projects/0b32a883f9ee4186b559190a6cddf33f/users/c6edb603589b4b1984b3b2a9df6c8151/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:56.938840 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3c1d9-8981-41c1-88b8-8db971fc3155 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:56.939142 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3c1d9-8981-41c1-88b8-8db971fc3155 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:56.939440 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3c1d9-8981-41c1-88b8-8db971fc3155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:56.967726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cda3c1d9-8981-41c1-88b8-8db971fc3155 None None] MFA Rules not processed for user `c6edb603589b4b1984b3b2a9df6c8151`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:56.969006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cda3c1d9-8981-41c1-88b8-8db971fc3155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:57.049238 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 869/1725] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:28:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:28:57.061030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1521ea-fbbe-4559-9ad3-2b6792332049 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:57.063042 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ef1521ea-fbbe-4559-9ad3-2b6792332049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:57.118751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1521ea-fbbe-4559-9ad3-2b6792332049 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:57.122387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e36873-bb23-48b3-a5b8-7b5009e0dcae service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37a9fea702cd405ea9fc14ec2c689648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:57.123616 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e36873-bb23-48b3-a5b8-7b5009e0dcae service glance] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:57.123902 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e36873-bb23-48b3-a5b8-7b5009e0dcae service glance] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:57.124174 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e36873-bb23-48b3-a5b8-7b5009e0dcae service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:57.132367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e36873-bb23-48b3-a5b8-7b5009e0dcae service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:57.133859 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e36873-bb23-48b3-a5b8-7b5009e0dcae service glance] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:57.142610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 857/1726] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:00.358556 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec112a29-2c8c-41a0-814f-e90d9b8f3563 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:00.359138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec112a29-2c8c-41a0-814f-e90d9b8f3563 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:00.359138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec112a29-2c8c-41a0-814f-e90d9b8f3563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:00.401124 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ec112a29-2c8c-41a0-814f-e90d9b8f3563 None None] MFA Rules not processed for user `4ddcf6306e704fe9ad5ed8779178ae67`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:00.402464 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ec112a29-2c8c-41a0-814f-e90d9b8f3563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:00.461685 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 870/1727] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 30 18:29:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:00.476472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e36873-bb23-48b3-a5b8-7b5009e0dcae service glance] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:00.479164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e9e36873-bb23-48b3-a5b8-7b5009e0dcae service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:00.560082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e36873-bb23-48b3-a5b8-7b5009e0dcae service glance] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:00.565396 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36105e4-4dfa-488f-ad5e-3a31cf7bc6dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bd521b9e93f4b3b8af64d46f36c019e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:00.566075 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36105e4-4dfa-488f-ad5e-3a31cf7bc6dc service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:00.566362 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36105e4-4dfa-488f-ad5e-3a31cf7bc6dc service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:00.566657 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36105e4-4dfa-488f-ad5e-3a31cf7bc6dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:00.571825 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:00.575813 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36105e4-4dfa-488f-ad5e-3a31cf7bc6dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:00.578293 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36105e4-4dfa-488f-ad5e-3a31cf7bc6dc service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:00.588162 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 858/1728] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 30 18:29:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 18:29:03.142333 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec112a29-2c8c-41a0-814f-e90d9b8f3563 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:03.144531 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ec112a29-2c8c-41a0-814f-e90d9b8f3563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:03.156493 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:03.266106 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec112a29-2c8c-41a0-814f-e90d9b8f3563 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:03.271950 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8bd031-7c6f-4221-8cb7-a5ea8ec96009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:03.273084 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8bd031-7c6f-4221-8cb7-a5ea8ec96009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:03.273463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8bd031-7c6f-4221-8cb7-a5ea8ec96009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:03.273922 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8bd031-7c6f-4221-8cb7-a5ea8ec96009 admin admin] PATH_INFO: `/v3/users/13907c48e3ea4b158b4b49fb7fbae2ee` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:03.275285 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8bd031-7c6f-4221-8cb7-a5ea8ec96009 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13907c48e3ea4b158b4b49fb7fbae2ee)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:03.277540 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8bd031-7c6f-4221-8cb7-a5ea8ec96009 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:03.380451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36105e4-4dfa-488f-ad5e-3a31cf7bc6dc service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:03.382432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a36105e4-4dfa-488f-ad5e-3a31cf7bc6dc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:03.406813 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36105e4-4dfa-488f-ad5e-3a31cf7bc6dc service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:03.419232 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e8532-8c77-4491-b56d-2db2b9df3797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:03.420890 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e8532-8c77-4491-b56d-2db2b9df3797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:03.421767 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e8532-8c77-4491-b56d-2db2b9df3797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:03.423130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e8532-8c77-4491-b56d-2db2b9df3797 admin admin] PATH_INFO: `/v3/users/46f996ff9e124483ad8dcc0163299159` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:03.426414 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e8532-8c77-4491-b56d-2db2b9df3797 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46f996ff9e124483ad8dcc0163299159)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:03.429104 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e8532-8c77-4491-b56d-2db2b9df3797 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:03.453228 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0f8bd031-7c6f-4221-8cb7-a5ea8ec96009 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13907c48e3ea4b158b4b49fb7fbae2ee'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:03.528342 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0f8bd031-7c6f-4221-8cb7-a5ea8ec96009 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13907c48e3ea4b158b4b49fb7fbae2ee'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:03.529924 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0f8bd031-7c6f-4221-8cb7-a5ea8ec96009 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13907c48e3ea4b158b4b49fb7fbae2ee'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:03.549038 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1d1e8532-8c77-4491-b56d-2db2b9df3797 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46f996ff9e124483ad8dcc0163299159'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:03.549707 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1d1e8532-8c77-4491-b56d-2db2b9df3797 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46f996ff9e124483ad8dcc0163299159'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:03.558256 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0f8bd031-7c6f-4221-8cb7-a5ea8ec96009 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13907c48e3ea4b158b4b49fb7fbae2ee'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:03.559087 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0f8bd031-7c6f-4221-8cb7-a5ea8ec96009 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13907c48e3ea4b158b4b49fb7fbae2ee'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:03.587930 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 871/1729] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:03 2022] DELETE /identity/v3/users/13907c48e3ea4b158b4b49fb7fbae2ee => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:03.592207 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1d1e8532-8c77-4491-b56d-2db2b9df3797 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46f996ff9e124483ad8dcc0163299159'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:03.592946 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6af7b-f283-4647-80d8-6d945387976a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:03.593244 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6af7b-f283-4647-80d8-6d945387976a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:03.593532 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6af7b-f283-4647-80d8-6d945387976a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:03.604672 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1d1e8532-8c77-4491-b56d-2db2b9df3797 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46f996ff9e124483ad8dcc0163299159'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:03.605361 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1d1e8532-8c77-4491-b56d-2db2b9df3797 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46f996ff9e124483ad8dcc0163299159'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:03.623361 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 859/1730] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:03 2022] DELETE /identity/v3/users/46f996ff9e124483ad8dcc0163299159 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:03.632790 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5135a-e86f-4d44-95ff-20c78dffc227 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:03.633707 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5135a-e86f-4d44-95ff-20c78dffc227 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:03.633707 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5135a-e86f-4d44-95ff-20c78dffc227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:03.639241 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7ac6af7b-f283-4647-80d8-6d945387976a None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:03.640517 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7ac6af7b-f283-4647-80d8-6d945387976a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:03.670126 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d7e5135a-e86f-4d44-95ff-20c78dffc227 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:03.673546 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d7e5135a-e86f-4d44-95ff-20c78dffc227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:03.797309 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 872/1731] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:03.827776 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 860/1732] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:03.871308 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6af7b-f283-4647-80d8-6d945387976a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:03.873286 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7ac6af7b-f283-4647-80d8-6d945387976a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:03.911831 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:03.951471 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5135a-e86f-4d44-95ff-20c78dffc227 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:03.953298 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d7e5135a-e86f-4d44-95ff-20c78dffc227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:03.970262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6af7b-f283-4647-80d8-6d945387976a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:03.971253 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5135a-e86f-4d44-95ff-20c78dffc227 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:03.974736 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc481c-1505-45a6-98d5-1c3db54ee081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:03.975800 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc481c-1505-45a6-98d5-1c3db54ee081 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:03.976343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc481c-1505-45a6-98d5-1c3db54ee081 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:03.977699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc481c-1505-45a6-98d5-1c3db54ee081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:03.982213 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4122f1-2a82-48b9-a077-9ced7ceae013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:03.983940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4122f1-2a82-48b9-a077-9ced7ceae013 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:03.983940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4122f1-2a82-48b9-a077-9ced7ceae013 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:03.985050 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4122f1-2a82-48b9-a077-9ced7ceae013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:04.008550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc481c-1505-45a6-98d5-1c3db54ee081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:04.012239 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc481c-1505-45a6-98d5-1c3db54ee081 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:04.019365 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4122f1-2a82-48b9-a077-9ced7ceae013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:04.023067 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4122f1-2a82-48b9-a077-9ced7ceae013 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:04.037780 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 873/1733] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:04.049900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 861/1734] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:29:04.792142 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc481c-1505-45a6-98d5-1c3db54ee081 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:04.793756 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-dccc481c-1505-45a6-98d5-1c3db54ee081 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:04.807703 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4122f1-2a82-48b9-a077-9ced7ceae013 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:04.809427 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ad4122f1-2a82-48b9-a077-9ced7ceae013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:04.827729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4122f1-2a82-48b9-a077-9ced7ceae013 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:04.833473 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f7582-2ce0-4cd0-8af8-3f4ede4fa005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:04.835035 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f7582-2ce0-4cd0-8af8-3f4ede4fa005 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:04.835497 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f7582-2ce0-4cd0-8af8-3f4ede4fa005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:04.835888 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f7582-2ce0-4cd0-8af8-3f4ede4fa005 admin admin] PATH_INFO: `/v3/projects/c83e905f79a242d6897cb0bbb4e94712` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:04.837192 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f7582-2ce0-4cd0-8af8-3f4ede4fa005 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c83e905f79a242d6897cb0bbb4e94712)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:04.839671 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f7582-2ce0-4cd0-8af8-3f4ede4fa005 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:04.845571 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc481c-1505-45a6-98d5-1c3db54ee081 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:04.852007 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2ce82-7c72-4d9e-acd4-ec293fcfe923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:04.853482 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2ce82-7c72-4d9e-acd4-ec293fcfe923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:04.853941 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2ce82-7c72-4d9e-acd4-ec293fcfe923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:04.854369 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2ce82-7c72-4d9e-acd4-ec293fcfe923 admin admin] PATH_INFO: `/v3/projects/58e2901bf4f847fcbdf60af88dd29207` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:04.855929 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c2ce82-7c72-4d9e-acd4-ec293fcfe923 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58e2901bf4f847fcbdf60af88dd29207)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:04.859111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c2ce82-7c72-4d9e-acd4-ec293fcfe923 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:04.907823 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:04.909976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4b6f7582-2ce0-4cd0-8af8-3f4ede4fa005 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c83e905f79a242d6897cb0bbb4e94712', 'request_id': 'req-4b6f7582-2ce0-4cd0-8af8-3f4ede4fa005'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:04.921415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4b6f7582-2ce0-4cd0-8af8-3f4ede4fa005 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c83e905f79a242d6897cb0bbb4e94712', 'request_id': 'req-4b6f7582-2ce0-4cd0-8af8-3f4ede4fa005'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:04.944839 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b1c2ce82-7c72-4d9e-acd4-ec293fcfe923 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58e2901bf4f847fcbdf60af88dd29207', 'request_id': 'req-b1c2ce82-7c72-4d9e-acd4-ec293fcfe923'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:04.956886 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4b6f7582-2ce0-4cd0-8af8-3f4ede4fa005 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c83e905f79a242d6897cb0bbb4e94712', 'request_id': 'req-4b6f7582-2ce0-4cd0-8af8-3f4ede4fa005'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:04.965348 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b1c2ce82-7c72-4d9e-acd4-ec293fcfe923 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58e2901bf4f847fcbdf60af88dd29207', 'request_id': 'req-b1c2ce82-7c72-4d9e-acd4-ec293fcfe923'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:04.974071 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b1c2ce82-7c72-4d9e-acd4-ec293fcfe923 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58e2901bf4f847fcbdf60af88dd29207', 'request_id': 'req-b1c2ce82-7c72-4d9e-acd4-ec293fcfe923'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:04.982870 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4b6f7582-2ce0-4cd0-8af8-3f4ede4fa005 admin admin] The token cache is being invalidate because project c83e905f79a242d6897cb0bbb4e94712 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:04.983192 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4b6f7582-2ce0-4cd0-8af8-3f4ede4fa005 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:04.984235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4b6f7582-2ce0-4cd0-8af8-3f4ede4fa005 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:04.986929 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 862/1735] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:04 2022] DELETE /identity/v3/projects/c83e905f79a242d6897cb0bbb4e94712 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:04.991932 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890f947-0fd1-44fb-965b-12bf30c08c15 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:04.992865 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890f947-0fd1-44fb-965b-12bf30c08c15 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:04.993326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890f947-0fd1-44fb-965b-12bf30c08c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:05.002985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b1c2ce82-7c72-4d9e-acd4-ec293fcfe923 admin admin] The token cache is being invalidate because project 58e2901bf4f847fcbdf60af88dd29207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:05.003503 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b1c2ce82-7c72-4d9e-acd4-ec293fcfe923 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:05.004479 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b1c2ce82-7c72-4d9e-acd4-ec293fcfe923 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:05.007524 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 874/1736] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:04 2022] DELETE /identity/v3/projects/58e2901bf4f847fcbdf60af88dd29207 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:05.014403 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed14e7a-16ac-48bb-95dc-8353d7126cd2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:05.014808 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed14e7a-16ac-48bb-95dc-8353d7126cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:05.016393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed14e7a-16ac-48bb-95dc-8353d7126cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:05.034723 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e890f947-0fd1-44fb-965b-12bf30c08c15 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:05.036063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e890f947-0fd1-44fb-965b-12bf30c08c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:05.056616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-eed14e7a-16ac-48bb-95dc-8353d7126cd2 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:05.058644 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-eed14e7a-16ac-48bb-95dc-8353d7126cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:05.112777 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 863/1737] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:05.119829 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca0b09-140f-45de-9625-ef2b09977f09 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:05.120298 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca0b09-140f-45de-9625-ef2b09977f09 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:05.122231 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca0b09-140f-45de-9625-ef2b09977f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:05.147529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8fca0b09-140f-45de-9625-ef2b09977f09 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:05.148851 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8fca0b09-140f-45de-9625-ef2b09977f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:05.161639 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 864/1738] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:05.165538 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca0b09-140f-45de-9625-ef2b09977f09 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:05.174014 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 875/1739] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:29:05.178121 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca0b09-140f-45de-9625-ef2b09977f09 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:05.183507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76d583-f178-4c84-a145-ae5b6d2d7ce9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:05.183805 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76d583-f178-4c84-a145-ae5b6d2d7ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:05.184087 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76d583-f178-4c84-a145-ae5b6d2d7ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:05.184590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31613bda-633d-4645-9b1f-84953c23df60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:05.185469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31613bda-633d-4645-9b1f-84953c23df60 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:05.186452 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31613bda-633d-4645-9b1f-84953c23df60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:05.186993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31613bda-633d-4645-9b1f-84953c23df60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:05.187789 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31613bda-633d-4645-9b1f-84953c23df60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:29:05.188101 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31613bda-633d-4645-9b1f-84953c23df60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:05.189783 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31613bda-633d-4645-9b1f-84953c23df60 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:05.208260 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 865/1740] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:29:05.208437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2c76d583-f178-4c84-a145-ae5b6d2d7ce9 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:05.208437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2c76d583-f178-4c84-a145-ae5b6d2d7ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:05.209435 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31613bda-633d-4645-9b1f-84953c23df60 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:05.215045 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 876/1741] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:29:05.219709 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76d583-f178-4c84-a145-ae5b6d2d7ce9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:05.222727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31613bda-633d-4645-9b1f-84953c23df60 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:05.227925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043c27a-7bfa-45a1-82a8-3d960df6de03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:05.228570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043c27a-7bfa-45a1-82a8-3d960df6de03 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:05.228853 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76d583-f178-4c84-a145-ae5b6d2d7ce9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:05.228945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043c27a-7bfa-45a1-82a8-3d960df6de03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:05.229137 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043c27a-7bfa-45a1-82a8-3d960df6de03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:05.229986 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043c27a-7bfa-45a1-82a8-3d960df6de03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:05.232040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043c27a-7bfa-45a1-82a8-3d960df6de03 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:05.233799 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7508e2-7b1e-40b6-929b-5c06722aa1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:05.234445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7508e2-7b1e-40b6-929b-5c06722aa1d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:05.234734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7508e2-7b1e-40b6-929b-5c06722aa1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:05.235016 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7508e2-7b1e-40b6-929b-5c06722aa1d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:05.235741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7508e2-7b1e-40b6-929b-5c06722aa1d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:29:05.236055 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7508e2-7b1e-40b6-929b-5c06722aa1d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:05.237672 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7508e2-7b1e-40b6-929b-5c06722aa1d3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:05.249359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 877/1742] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:29:05.254091 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7508e2-7b1e-40b6-929b-5c06722aa1d3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:05.262413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 866/1743] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:05 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:05.263721 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043c27a-7bfa-45a1-82a8-3d960df6de03 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:05.265450 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7508e2-7b1e-40b6-929b-5c06722aa1d3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:05.295166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:05.324004 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72073237-2ea5-4c8b-b26f-9fd3203e0dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:05.324700 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72073237-2ea5-4c8b-b26f-9fd3203e0dec admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:05.325067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72073237-2ea5-4c8b-b26f-9fd3203e0dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:05.325426 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72073237-2ea5-4c8b-b26f-9fd3203e0dec admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:05.326742 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72073237-2ea5-4c8b-b26f-9fd3203e0dec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:05.326892 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043c27a-7bfa-45a1-82a8-3d960df6de03 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:05.329199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72073237-2ea5-4c8b-b26f-9fd3203e0dec admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:05.332047 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ca1cd-ef4c-41fd-a10e-ecd15296a2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:05.332694 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ca1cd-ef4c-41fd-a10e-ecd15296a2df admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:05.332975 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ca1cd-ef4c-41fd-a10e-ecd15296a2df admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:05.333248 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ca1cd-ef4c-41fd-a10e-ecd15296a2df admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:05.334109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ca1cd-ef4c-41fd-a10e-ecd15296a2df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:05.336016 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ca1cd-ef4c-41fd-a10e-ecd15296a2df admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:05.359967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 878/1744] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:05 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:29:05.360144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 867/1745] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:05 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:29:05.360939 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ca1cd-ef4c-41fd-a10e-ecd15296a2df admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:05.361044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72073237-2ea5-4c8b-b26f-9fd3203e0dec admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:05.370696 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ca1cd-ef4c-41fd-a10e-ecd15296a2df admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:05.390659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:05.425373 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72073237-2ea5-4c8b-b26f-9fd3203e0dec admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:05.428881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a9080-557e-4d20-8e6b-a1a7207c11dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:05.429572 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a9080-557e-4d20-8e6b-a1a7207c11dc admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:05.429945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a9080-557e-4d20-8e6b-a1a7207c11dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:05.430225 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a9080-557e-4d20-8e6b-a1a7207c11dc admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:05.430657 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b39c0-d3a7-446b-b94e-747e47ccd7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:05.430915 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a9080-557e-4d20-8e6b-a1a7207c11dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:05.431323 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b39c0-d3a7-446b-b94e-747e47ccd7ea admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:05.431620 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b39c0-d3a7-446b-b94e-747e47ccd7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:05.431928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b39c0-d3a7-446b-b94e-747e47ccd7ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:05.432662 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b39c0-d3a7-446b-b94e-747e47ccd7ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:05.433003 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a9080-557e-4d20-8e6b-a1a7207c11dc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:05.434422 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b39c0-d3a7-446b-b94e-747e47ccd7ea admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:05.449303 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 879/1746] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:05.463877 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b39c0-d3a7-446b-b94e-747e47ccd7ea admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:05.478617 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b39c0-d3a7-446b-b94e-747e47ccd7ea admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:05.485962 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde362a8-1c54-446d-bb26-e44453694ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:05.486812 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde362a8-1c54-446d-bb26-e44453694ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:05.488513 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 868/1747] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:05 2022] POST /identity/v3/users => generated 286 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:05.488655 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde362a8-1c54-446d-bb26-e44453694ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:05.488655 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde362a8-1c54-446d-bb26-e44453694ff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:05.490136 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde362a8-1c54-446d-bb26-e44453694ff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:05.491838 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a9080-557e-4d20-8e6b-a1a7207c11dc admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:05.491992 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde362a8-1c54-446d-bb26-e44453694ff9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:05.507800 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a9080-557e-4d20-8e6b-a1a7207c11dc admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:05.511840 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 880/1748] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:05.516361 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde362a8-1c54-446d-bb26-e44453694ff9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:05.517757 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8810247f-769c-4d58-92e7-621a3a0c25c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:05.519476 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8810247f-769c-4d58-92e7-621a3a0c25c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:05.519960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8810247f-769c-4d58-92e7-621a3a0c25c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:05.520524 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8810247f-769c-4d58-92e7-621a3a0c25c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:05.521435 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8810247f-769c-4d58-92e7-621a3a0c25c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:05.524550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8810247f-769c-4d58-92e7-621a3a0c25c6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:05.527865 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde362a8-1c54-446d-bb26-e44453694ff9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:05.539085 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 869/1749] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:05.540242 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8810247f-769c-4d58-92e7-621a3a0c25c6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:05.542270 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d328a-d300-4798-9c9b-bb7ffd88a903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:05.543241 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d328a-d300-4798-9c9b-bb7ffd88a903 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:05.543527 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d328a-d300-4798-9c9b-bb7ffd88a903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:05.543808 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d328a-d300-4798-9c9b-bb7ffd88a903 admin admin] PATH_INFO: `/v3/projects/c9fe62c5a48540d69e796f365be164d8/users/474e525a853045b3a526b09f4124f032/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:05.553046 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8810247f-769c-4d58-92e7-621a3a0c25c6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:05.559823 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec526de1-156c-4a78-a8a7-b322137a4ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:05.561114 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec526de1-156c-4a78-a8a7-b322137a4ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:05.561463 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec526de1-156c-4a78-a8a7-b322137a4ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:05.562054 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec526de1-156c-4a78-a8a7-b322137a4ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:05.562790 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec526de1-156c-4a78-a8a7-b322137a4ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:05.563212 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d328a-d300-4798-9c9b-bb7ffd88a903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9fe62c5a48540d69e796f365be164d8, user_id=474e525a853045b3a526b09f4124f032, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:05.564462 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec526de1-156c-4a78-a8a7-b322137a4ab0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:05.566595 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d328a-d300-4798-9c9b-bb7ffd88a903 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:05.577711 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 870/1750] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:05.582188 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec526de1-156c-4a78-a8a7-b322137a4ab0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:05.583579 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 881/1751] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:05 2022] PUT /identity/v3/projects/c9fe62c5a48540d69e796f365be164d8/users/474e525a853045b3a526b09f4124f032/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:05.593683 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec526de1-156c-4a78-a8a7-b322137a4ab0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:05.594237 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66807553-679a-4d86-bc32-1e65ca2ef4d1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:05.594536 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66807553-679a-4d86-bc32-1e65ca2ef4d1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:05.594821 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66807553-679a-4d86-bc32-1e65ca2ef4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:05.625515 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-66807553-679a-4d86-bc32-1e65ca2ef4d1 None None] MFA Rules not processed for user `474e525a853045b3a526b09f4124f032`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:05.627291 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-66807553-679a-4d86-bc32-1e65ca2ef4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:05.666109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab48389-7802-4e4b-a06a-75869538d81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:05.667269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab48389-7802-4e4b-a06a-75869538d81a admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:05.667656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab48389-7802-4e4b-a06a-75869538d81a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:05.668040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab48389-7802-4e4b-a06a-75869538d81a admin admin] PATH_INFO: `/v3/projects/c0c4082a62c14d55852f870e6ca9e477/users/5dc3371296fe49d5b11520efda362a5f/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:05.687314 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab48389-7802-4e4b-a06a-75869538d81a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0c4082a62c14d55852f870e6ca9e477, user_id=5dc3371296fe49d5b11520efda362a5f, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:05.690431 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab48389-7802-4e4b-a06a-75869538d81a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:05.723881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 871/1752] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:05 2022] PUT /identity/v3/projects/c0c4082a62c14d55852f870e6ca9e477/users/5dc3371296fe49d5b11520efda362a5f/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:05.728017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 882/1753] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:29:05.735052 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5dc204-e6a7-4439-b94e-2a11d13be2d1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:05.735345 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5dc204-e6a7-4439-b94e-2a11d13be2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:05.735629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5dc204-e6a7-4439-b94e-2a11d13be2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:05.740223 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66807553-679a-4d86-bc32-1e65ca2ef4d1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:05.741772 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-66807553-679a-4d86-bc32-1e65ca2ef4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:05.753798 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:05.773155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fd5dc204-e6a7-4439-b94e-2a11d13be2d1 None None] MFA Rules not processed for user `5dc3371296fe49d5b11520efda362a5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:05.774440 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fd5dc204-e6a7-4439-b94e-2a11d13be2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:05.827083 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66807553-679a-4d86-bc32-1e65ca2ef4d1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:05.845308 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 872/1754] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:05.849791 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5dc204-e6a7-4439-b94e-2a11d13be2d1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:05.855229 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:05.892163 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5865c-5e12-45d5-94ea-88024a484c59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:05.892811 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5865c-5e12-45d5-94ea-88024a484c59 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:05.893098 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5865c-5e12-45d5-94ea-88024a484c59 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:05.893383 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5865c-5e12-45d5-94ea-88024a484c59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:05.915099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5dc204-e6a7-4439-b94e-2a11d13be2d1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:05.920257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95548c8f-c68e-4f50-a7a6-b1c8a62f3387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:05.921116 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95548c8f-c68e-4f50-a7a6-b1c8a62f3387 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:05.921401 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95548c8f-c68e-4f50-a7a6-b1c8a62f3387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:05.921711 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95548c8f-c68e-4f50-a7a6-b1c8a62f3387 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:05.922397 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95548c8f-c68e-4f50-a7a6-b1c8a62f3387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:05.924313 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95548c8f-c68e-4f50-a7a6-b1c8a62f3387 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:05.950076 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 873/1755] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:05 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:29:05.951950 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5865c-5e12-45d5-94ea-88024a484c59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:05.953607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5865c-5e12-45d5-94ea-88024a484c59 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:05.955068 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95548c8f-c68e-4f50-a7a6-b1c8a62f3387 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:05.963010 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 883/1756] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:29:06.050399 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95548c8f-c68e-4f50-a7a6-b1c8a62f3387 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:06.055545 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e564b0-17b8-44e1-8772-c20b571ce5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:06.056347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e564b0-17b8-44e1-8772-c20b571ce5be admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:06.056706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e564b0-17b8-44e1-8772-c20b571ce5be admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:06.057077 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e564b0-17b8-44e1-8772-c20b571ce5be admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:06.058087 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e564b0-17b8-44e1-8772-c20b571ce5be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:06.060923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e564b0-17b8-44e1-8772-c20b571ce5be admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:06.099612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 874/1757] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:05 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:29:06.102264 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5865c-5e12-45d5-94ea-88024a484c59 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:06.115833 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5865c-5e12-45d5-94ea-88024a484c59 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:06.120564 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9be9e-d5a3-4d5b-898e-aeb599d5250c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:06.121303 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9be9e-d5a3-4d5b-898e-aeb599d5250c admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:06.121809 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9be9e-d5a3-4d5b-898e-aeb599d5250c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:06.122145 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9be9e-d5a3-4d5b-898e-aeb599d5250c admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:06.127628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa9be9e-d5a3-4d5b-898e-aeb599d5250c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:06.127628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa9be9e-d5a3-4d5b-898e-aeb599d5250c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:06.137564 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 884/1758] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:06.141564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e564b0-17b8-44e1-8772-c20b571ce5be admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:06.151633 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e564b0-17b8-44e1-8772-c20b571ce5be admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:06.156380 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4873448-35c3-4259-a43a-537b0182c7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:06.157465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4873448-35c3-4259-a43a-537b0182c7eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:06.157852 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4873448-35c3-4259-a43a-537b0182c7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:06.158134 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4873448-35c3-4259-a43a-537b0182c7eb admin admin] PATH_INFO: `/v3/projects/c14847b9cb394a778ff934ec6168dd8a/users/baa7354ee46143dc846d99bcd0137cc3/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:06.184124 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4873448-35c3-4259-a43a-537b0182c7eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c14847b9cb394a778ff934ec6168dd8a, user_id=baa7354ee46143dc846d99bcd0137cc3, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:06.187889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4873448-35c3-4259-a43a-537b0182c7eb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:06.205126 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 875/1759] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:06 2022] PUT /identity/v3/projects/c14847b9cb394a778ff934ec6168dd8a/users/baa7354ee46143dc846d99bcd0137cc3/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:06.209846 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9be9e-d5a3-4d5b-898e-aeb599d5250c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:06.229856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4873448-35c3-4259-a43a-537b0182c7eb admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:06.233831 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9be9e-d5a3-4d5b-898e-aeb599d5250c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:06.286272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:06.316924 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe0e2d-581e-49f4-9402-743fe3223e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:06.317641 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe0e2d-581e-49f4-9402-743fe3223e74 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:06.317927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe0e2d-581e-49f4-9402-743fe3223e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:06.318205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe0e2d-581e-49f4-9402-743fe3223e74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:06.318926 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe0e2d-581e-49f4-9402-743fe3223e74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:06.320558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe0e2d-581e-49f4-9402-743fe3223e74 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:06.323252 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:06.330540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4873448-35c3-4259-a43a-537b0182c7eb admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:06.333348 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 885/1760] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:06.337390 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe0e2d-581e-49f4-9402-743fe3223e74 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:06.339424 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43979388-6e6b-418e-9985-5e4abef407d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:06.340756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43979388-6e6b-418e-9985-5e4abef407d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:06.341223 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43979388-6e6b-418e-9985-5e4abef407d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:06.342322 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43979388-6e6b-418e-9985-5e4abef407d1 admin admin] PATH_INFO: `/v3/users/474e525a853045b3a526b09f4124f032` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:06.343933 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43979388-6e6b-418e-9985-5e4abef407d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=474e525a853045b3a526b09f4124f032)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:06.346972 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43979388-6e6b-418e-9985-5e4abef407d1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:06.348282 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe0e2d-581e-49f4-9402-743fe3223e74 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:06.353459 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b7c38d-8b49-4d57-9c3d-f22995f3585e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:06.354409 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b7c38d-8b49-4d57-9c3d-f22995f3585e admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:06.354703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b7c38d-8b49-4d57-9c3d-f22995f3585e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:06.354972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b7c38d-8b49-4d57-9c3d-f22995f3585e admin admin] PATH_INFO: `/v3/projects/c14847b9cb394a778ff934ec6168dd8a/users/baa7354ee46143dc846d99bcd0137cc3/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:06.357056 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b7c38d-8b49-4d57-9c3d-f22995f3585e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c14847b9cb394a778ff934ec6168dd8a, user_id=baa7354ee46143dc846d99bcd0137cc3, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:06.360297 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b7c38d-8b49-4d57-9c3d-f22995f3585e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:06.372260 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 886/1761] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:06 2022] PUT /identity/v3/projects/c14847b9cb394a778ff934ec6168dd8a/users/baa7354ee46143dc846d99bcd0137cc3/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:06.382724 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287e5c5-c61c-4d70-a681-d6acb76da9f7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:06.383008 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287e5c5-c61c-4d70-a681-d6acb76da9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:06.383330 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287e5c5-c61c-4d70-a681-d6acb76da9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:06.399311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-43979388-6e6b-418e-9985-5e4abef407d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '474e525a853045b3a526b09f4124f032'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:06.399994 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-43979388-6e6b-418e-9985-5e4abef407d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '474e525a853045b3a526b09f4124f032'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:06.410253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c287e5c5-c61c-4d70-a681-d6acb76da9f7 None None] MFA Rules not processed for user `baa7354ee46143dc846d99bcd0137cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:06.411489 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c287e5c5-c61c-4d70-a681-d6acb76da9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:06.412858 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-43979388-6e6b-418e-9985-5e4abef407d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '474e525a853045b3a526b09f4124f032'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:06.421879 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-43979388-6e6b-418e-9985-5e4abef407d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '474e525a853045b3a526b09f4124f032'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:06.422527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-43979388-6e6b-418e-9985-5e4abef407d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '474e525a853045b3a526b09f4124f032'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:06.435008 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 876/1762] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:06 2022] DELETE /identity/v3/users/474e525a853045b3a526b09f4124f032 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:06.439968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fb14a-5076-4b00-b196-33d78545504a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:06.440297 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fb14a-5076-4b00-b196-33d78545504a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:06.440580 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fb14a-5076-4b00-b196-33d78545504a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:06.460611 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f48fb14a-5076-4b00-b196-33d78545504a None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:06.461849 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f48fb14a-5076-4b00-b196-33d78545504a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:06.480048 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 887/1763] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:29:06.492281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287e5c5-c61c-4d70-a681-d6acb76da9f7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:06.493698 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c287e5c5-c61c-4d70-a681-d6acb76da9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:06.522476 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 877/1764] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:29:06.532764 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fb14a-5076-4b00-b196-33d78545504a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:06.534189 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f48fb14a-5076-4b00-b196-33d78545504a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:06.552742 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287e5c5-c61c-4d70-a681-d6acb76da9f7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:06.595548 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fb14a-5076-4b00-b196-33d78545504a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:06.599416 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d1f8f-d45b-4ea5-b960-e9f9d8b5cf7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:06.600015 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d1f8f-d45b-4ea5-b960-e9f9d8b5cf7d service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:06.600300 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d1f8f-d45b-4ea5-b960-e9f9d8b5cf7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:06.600579 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d1f8f-d45b-4ea5-b960-e9f9d8b5cf7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:06.611017 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d1f8f-d45b-4ea5-b960-e9f9d8b5cf7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:06.612400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d1f8f-d45b-4ea5-b960-e9f9d8b5cf7d service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:06.624041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 878/1765] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:06.743901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b9d70-ea3e-413d-b9b3-daee68d6052d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:06.744946 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b9d70-ea3e-413d-b9b3-daee68d6052d service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:06.745308 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b9d70-ea3e-413d-b9b3-daee68d6052d service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:06.745535 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b9d70-ea3e-413d-b9b3-daee68d6052d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:06.756362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:06.815872 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b9d70-ea3e-413d-b9b3-daee68d6052d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:06.817383 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b9d70-ea3e-413d-b9b3-daee68d6052d service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:06.836963 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 888/1766] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:06 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:07.014549 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d1f8f-d45b-4ea5-b960-e9f9d8b5cf7d service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:07.016079 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c35d1f8f-d45b-4ea5-b960-e9f9d8b5cf7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:07.031568 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d1f8f-d45b-4ea5-b960-e9f9d8b5cf7d service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:07.048071 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a268d4-5ad1-428c-9a33-0f823cc97a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:07.049001 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a268d4-5ad1-428c-9a33-0f823cc97a74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:07.049224 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a268d4-5ad1-428c-9a33-0f823cc97a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:07.049518 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a268d4-5ad1-428c-9a33-0f823cc97a74 admin admin] PATH_INFO: `/v3/projects/c9fe62c5a48540d69e796f365be164d8` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:07.051440 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a268d4-5ad1-428c-9a33-0f823cc97a74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9fe62c5a48540d69e796f365be164d8)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:07.053362 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a268d4-5ad1-428c-9a33-0f823cc97a74 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:07.071888 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:07.125945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46a268d4-5ad1-428c-9a33-0f823cc97a74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9fe62c5a48540d69e796f365be164d8', 'request_id': 'req-46a268d4-5ad1-428c-9a33-0f823cc97a74'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:07.135766 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46a268d4-5ad1-428c-9a33-0f823cc97a74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9fe62c5a48540d69e796f365be164d8', 'request_id': 'req-46a268d4-5ad1-428c-9a33-0f823cc97a74'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:07.146903 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46a268d4-5ad1-428c-9a33-0f823cc97a74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9fe62c5a48540d69e796f365be164d8', 'request_id': 'req-46a268d4-5ad1-428c-9a33-0f823cc97a74'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:07.163552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46a268d4-5ad1-428c-9a33-0f823cc97a74 admin admin] The token cache is being invalidate because project c9fe62c5a48540d69e796f365be164d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:07.163851 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46a268d4-5ad1-428c-9a33-0f823cc97a74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:07.164503 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-46a268d4-5ad1-428c-9a33-0f823cc97a74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:07.166442 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 879/1767] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:07 2022] DELETE /identity/v3/projects/c9fe62c5a48540d69e796f365be164d8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:07.173774 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f950e-0896-4fc2-a860-3c69e5ac593d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:07.174157 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f950e-0896-4fc2-a860-3c69e5ac593d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:07.174589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f950e-0896-4fc2-a860-3c69e5ac593d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:07.199794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0b8f950e-0896-4fc2-a860-3c69e5ac593d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:07.202862 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0b8f950e-0896-4fc2-a860-3c69e5ac593d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:07.282844 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 889/1768] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:07.292240 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de557db6-1d56-4a95-8ec4-e5f55f7c9378 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:07.292591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de557db6-1d56-4a95-8ec4-e5f55f7c9378 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:07.292985 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de557db6-1d56-4a95-8ec4-e5f55f7c9378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:07.313314 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-de557db6-1d56-4a95-8ec4-e5f55f7c9378 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:07.314739 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-de557db6-1d56-4a95-8ec4-e5f55f7c9378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:07.322996 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 880/1769] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:07.331471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f950e-0896-4fc2-a860-3c69e5ac593d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:07.333421 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de557db6-1d56-4a95-8ec4-e5f55f7c9378 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:07.334815 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-de557db6-1d56-4a95-8ec4-e5f55f7c9378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:07.337838 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f950e-0896-4fc2-a860-3c69e5ac593d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:07.343021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a3fda-3c89-4740-a2a1-750739e18a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:07.343689 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a3fda-3c89-4740-a2a1-750739e18a25 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:07.343976 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a3fda-3c89-4740-a2a1-750739e18a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:07.344258 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a3fda-3c89-4740-a2a1-750739e18a25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:07.345049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a3fda-3c89-4740-a2a1-750739e18a25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:29:07.345297 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a3fda-3c89-4740-a2a1-750739e18a25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:07.346966 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a3fda-3c89-4740-a2a1-750739e18a25 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:07.347510 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de557db6-1d56-4a95-8ec4-e5f55f7c9378 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:07.352414 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9513a-a423-4348-96eb-6730c02fe11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:07.353304 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9513a-a423-4348-96eb-6730c02fe11d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:07.353606 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9513a-a423-4348-96eb-6730c02fe11d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:07.353919 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9513a-a423-4348-96eb-6730c02fe11d admin admin] PATH_INFO: `/v3/users/5dc3371296fe49d5b11520efda362a5f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:07.355120 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9513a-a423-4348-96eb-6730c02fe11d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dc3371296fe49d5b11520efda362a5f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:07.357861 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9513a-a423-4348-96eb-6730c02fe11d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:07.359159 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 890/1770] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:29:07.363818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a3fda-3c89-4740-a2a1-750739e18a25 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:07.376579 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a3fda-3c89-4740-a2a1-750739e18a25 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:07.381347 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc934b7-99e7-4be3-b63a-c10ae741d157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:07.382058 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc934b7-99e7-4be3-b63a-c10ae741d157 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:07.382750 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc934b7-99e7-4be3-b63a-c10ae741d157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:07.383555 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc934b7-99e7-4be3-b63a-c10ae741d157 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:07.384921 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc934b7-99e7-4be3-b63a-c10ae741d157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:07.387160 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc934b7-99e7-4be3-b63a-c10ae741d157 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:07.416678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 891/1771] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:07 2022] POST /identity/v3/projects => generated 409 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:29:07.421109 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc934b7-99e7-4be3-b63a-c10ae741d157 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:07.421697 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-75f9513a-a423-4348-96eb-6730c02fe11d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dc3371296fe49d5b11520efda362a5f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:07.422690 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-75f9513a-a423-4348-96eb-6730c02fe11d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dc3371296fe49d5b11520efda362a5f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:07.454674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-75f9513a-a423-4348-96eb-6730c02fe11d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dc3371296fe49d5b11520efda362a5f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:07.468369 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-75f9513a-a423-4348-96eb-6730c02fe11d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dc3371296fe49d5b11520efda362a5f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:07.469026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-75f9513a-a423-4348-96eb-6730c02fe11d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dc3371296fe49d5b11520efda362a5f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:07.484101 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 881/1772] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:07 2022] DELETE /identity/v3/users/5dc3371296fe49d5b11520efda362a5f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:07.488398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06126de1-5ed1-4f32-8bd2-550ec5034938 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:07.488699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06126de1-5ed1-4f32-8bd2-550ec5034938 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:07.488982 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06126de1-5ed1-4f32-8bd2-550ec5034938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:07.507406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-06126de1-5ed1-4f32-8bd2-550ec5034938 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:07.508637 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-06126de1-5ed1-4f32-8bd2-550ec5034938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:07.515509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc934b7-99e7-4be3-b63a-c10ae741d157 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:07.536751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:07.586239 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aab908-04a9-40a1-b2f5-7b53c39a81c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:07.586604 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 882/1773] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:29:07.587169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aab908-04a9-40a1-b2f5-7b53c39a81c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:07.587545 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aab908-04a9-40a1-b2f5-7b53c39a81c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:07.587838 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aab908-04a9-40a1-b2f5-7b53c39a81c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:07.588537 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aab908-04a9-40a1-b2f5-7b53c39a81c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:07.590674 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aab908-04a9-40a1-b2f5-7b53c39a81c2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:07.597364 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06126de1-5ed1-4f32-8bd2-550ec5034938 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:07.598841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-06126de1-5ed1-4f32-8bd2-550ec5034938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:07.610962 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:07.612688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 892/1774] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:07 2022] POST /identity/v3/users => generated 305 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:29:07.616582 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aab908-04a9-40a1-b2f5-7b53c39a81c2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:07.618388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f6aab908-04a9-40a1-b2f5-7b53c39a81c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:07.630664 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aab908-04a9-40a1-b2f5-7b53c39a81c2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:07.635765 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5fd466-bdce-468d-b0cc-023f3eb90c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:07.636457 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fd466-bdce-468d-b0cc-023f3eb90c84 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:07.636748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fd466-bdce-468d-b0cc-023f3eb90c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:07.637065 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fd466-bdce-468d-b0cc-023f3eb90c84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:07.637920 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5fd466-bdce-468d-b0cc-023f3eb90c84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:07.640214 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5fd466-bdce-468d-b0cc-023f3eb90c84 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:07.650968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 893/1775] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:29:07.654836 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5fd466-bdce-468d-b0cc-023f3eb90c84 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:07.663143 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06126de1-5ed1-4f32-8bd2-550ec5034938 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:07.664219 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5fd466-bdce-468d-b0cc-023f3eb90c84 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:07.666796 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cadeeea-0fbd-44a0-a2e5-97830b11d058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:07.667440 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadeeea-0fbd-44a0-a2e5-97830b11d058 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:07.667727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadeeea-0fbd-44a0-a2e5-97830b11d058 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:07.668008 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadeeea-0fbd-44a0-a2e5-97830b11d058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:07.668875 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d1376-b0af-4337-98ff-355357b4b6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:07.670058 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d1376-b0af-4337-98ff-355357b4b6a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:07.670688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d1376-b0af-4337-98ff-355357b4b6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:07.670688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d1376-b0af-4337-98ff-355357b4b6a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:07.671324 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d1376-b0af-4337-98ff-355357b4b6a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:07.672901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d1376-b0af-4337-98ff-355357b4b6a2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:07.678269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cadeeea-0fbd-44a0-a2e5-97830b11d058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:07.679706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cadeeea-0fbd-44a0-a2e5-97830b11d058 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:07.683431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 894/1776] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:07.688767 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d1376-b0af-4337-98ff-355357b4b6a2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:07.690037 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 883/1777] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:29:07.698794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d1376-b0af-4337-98ff-355357b4b6a2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:07.705463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bdff3f-9ec2-4896-a122-de53004322d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:07.706350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bdff3f-9ec2-4896-a122-de53004322d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:07.706690 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bdff3f-9ec2-4896-a122-de53004322d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:07.706913 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bdff3f-9ec2-4896-a122-de53004322d8 admin admin] PATH_INFO: `/v3/projects/2b032dfca7f44f13a41d06ff5346734e/users/8bafad298e2247b399e0787600908794/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:07.724519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bdff3f-9ec2-4896-a122-de53004322d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b032dfca7f44f13a41d06ff5346734e, user_id=8bafad298e2247b399e0787600908794, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:07.727256 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bdff3f-9ec2-4896-a122-de53004322d8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:07.743973 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 895/1778] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:07 2022] PUT /identity/v3/projects/2b032dfca7f44f13a41d06ff5346734e/users/8bafad298e2247b399e0787600908794/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:07.755454 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2024d-e18b-47b9-9051-92ecc48c2698 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:07.755749 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2024d-e18b-47b9-9051-92ecc48c2698 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:07.756028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2024d-e18b-47b9-9051-92ecc48c2698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:07.785301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-53c2024d-e18b-47b9-9051-92ecc48c2698 None None] MFA Rules not processed for user `8bafad298e2247b399e0787600908794`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:07.786701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-53c2024d-e18b-47b9-9051-92ecc48c2698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:07.868355 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 884/1779] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:07 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:07.873356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bdff3f-9ec2-4896-a122-de53004322d8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:07.934898 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bdff3f-9ec2-4896-a122-de53004322d8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:07.940589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba7327-d3f5-4b0c-8d98-6b140e09d044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:07.941617 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba7327-d3f5-4b0c-8d98-6b140e09d044 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:07.941617 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba7327-d3f5-4b0c-8d98-6b140e09d044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:07.941839 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba7327-d3f5-4b0c-8d98-6b140e09d044 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:07.942509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba7327-d3f5-4b0c-8d98-6b140e09d044 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:07.944423 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba7327-d3f5-4b0c-8d98-6b140e09d044 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:07.969776 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c2024d-e18b-47b9-9051-92ecc48c2698 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:07.971317 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-53c2024d-e18b-47b9-9051-92ecc48c2698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:07.972528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 896/1780] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:07 2022] POST /identity/v3/projects => generated 407 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:07.976411 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba7327-d3f5-4b0c-8d98-6b140e09d044 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:07.988509 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c2024d-e18b-47b9-9051-92ecc48c2698 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:08.040965 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:08.054973 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09aaf8-0828-4ec0-b59c-f16b32681082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:08.055853 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09aaf8-0828-4ec0-b59c-f16b32681082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:08.056144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09aaf8-0828-4ec0-b59c-f16b32681082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:08.056428 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09aaf8-0828-4ec0-b59c-f16b32681082 admin admin] PATH_INFO: `/v3/projects/c0c4082a62c14d55852f870e6ca9e477` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:08.057946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09aaf8-0828-4ec0-b59c-f16b32681082 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0c4082a62c14d55852f870e6ca9e477)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:08.060020 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09aaf8-0828-4ec0-b59c-f16b32681082 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:08.065631 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba7327-d3f5-4b0c-8d98-6b140e09d044 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:08.078295 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c722f-9c7a-4e38-96e3-f6a47bd7cdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:08.079358 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c722f-9c7a-4e38-96e3-f6a47bd7cdaf admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:08.079805 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c722f-9c7a-4e38-96e3-f6a47bd7cdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:08.080237 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c722f-9c7a-4e38-96e3-f6a47bd7cdaf admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:08.081096 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c722f-9c7a-4e38-96e3-f6a47bd7cdaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:08.085617 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c722f-9c7a-4e38-96e3-f6a47bd7cdaf admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:08.122514 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 897/1781] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:07 2022] POST /identity/v3/users => generated 310 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:29:08.127252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c722f-9c7a-4e38-96e3-f6a47bd7cdaf admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:08.131343 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:08.137546 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c722f-9c7a-4e38-96e3-f6a47bd7cdaf admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:08.140765 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bb09aaf8-0828-4ec0-b59c-f16b32681082 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0c4082a62c14d55852f870e6ca9e477', 'request_id': 'req-bb09aaf8-0828-4ec0-b59c-f16b32681082'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:08.148485 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bb09aaf8-0828-4ec0-b59c-f16b32681082 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0c4082a62c14d55852f870e6ca9e477', 'request_id': 'req-bb09aaf8-0828-4ec0-b59c-f16b32681082'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:08.167190 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bb09aaf8-0828-4ec0-b59c-f16b32681082 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0c4082a62c14d55852f870e6ca9e477', 'request_id': 'req-bb09aaf8-0828-4ec0-b59c-f16b32681082'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:08.188309 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bb09aaf8-0828-4ec0-b59c-f16b32681082 admin admin] The token cache is being invalidate because project c0c4082a62c14d55852f870e6ca9e477 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:08.188634 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bb09aaf8-0828-4ec0-b59c-f16b32681082 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:08.189451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bb09aaf8-0828-4ec0-b59c-f16b32681082 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:08.191489 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 885/1782] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:07 2022] DELETE /identity/v3/projects/c0c4082a62c14d55852f870e6ca9e477 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:08.195294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09aaf8-0828-4ec0-b59c-f16b32681082 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:08.197764 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bb09aaf8-0828-4ec0-b59c-f16b32681082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:08.229688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279826ac-2975-48db-b518-5f51c5f679ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:08.230542 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279826ac-2975-48db-b518-5f51c5f679ca admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:08.230976 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279826ac-2975-48db-b518-5f51c5f679ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:08.231449 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279826ac-2975-48db-b518-5f51c5f679ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:08.233290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279826ac-2975-48db-b518-5f51c5f679ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:08.235573 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279826ac-2975-48db-b518-5f51c5f679ca admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:08.256974 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 898/1783] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:29:08.260013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279826ac-2975-48db-b518-5f51c5f679ca admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:08.262395 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-279826ac-2975-48db-b518-5f51c5f679ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:08.289552 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279826ac-2975-48db-b518-5f51c5f679ca admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:08.308774 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09aaf8-0828-4ec0-b59c-f16b32681082 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:08.323309 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c4e90-6fde-46c5-a738-340d5b88bc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:08.325321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c4e90-6fde-46c5-a738-340d5b88bc0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:08.328461 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d0ee9-b46e-4e2c-8b12-bbb370faf9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:08.330451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d0ee9-b46e-4e2c-8b12-bbb370faf9e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:08.330937 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d0ee9-b46e-4e2c-8b12-bbb370faf9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:08.331868 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d0ee9-b46e-4e2c-8b12-bbb370faf9e5 admin admin] PATH_INFO: `/v3/projects/433968dd9da34d889fbfaa517bf22e09/users/0cb606739bfb4c78aa9dac40cecc26ae/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:08.333832 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c4e90-6fde-46c5-a738-340d5b88bc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:08.334418 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c4e90-6fde-46c5-a738-340d5b88bc0f admin admin] PATH_INFO: `/v3/users/baa7354ee46143dc846d99bcd0137cc3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:08.336102 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c4e90-6fde-46c5-a738-340d5b88bc0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=baa7354ee46143dc846d99bcd0137cc3)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:08.338239 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c4e90-6fde-46c5-a738-340d5b88bc0f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:08.351529 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d0ee9-b46e-4e2c-8b12-bbb370faf9e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=433968dd9da34d889fbfaa517bf22e09, user_id=0cb606739bfb4c78aa9dac40cecc26ae, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:08.355090 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d0ee9-b46e-4e2c-8b12-bbb370faf9e5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:08.376634 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 899/1784] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:08 2022] PUT /identity/v3/projects/433968dd9da34d889fbfaa517bf22e09/users/0cb606739bfb4c78aa9dac40cecc26ae/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:08.380536 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d0ee9-b46e-4e2c-8b12-bbb370faf9e5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:08.406887 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-737c4e90-6fde-46c5-a738-340d5b88bc0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baa7354ee46143dc846d99bcd0137cc3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:08.407673 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-737c4e90-6fde-46c5-a738-340d5b88bc0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baa7354ee46143dc846d99bcd0137cc3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:08.426997 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-737c4e90-6fde-46c5-a738-340d5b88bc0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baa7354ee46143dc846d99bcd0137cc3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:08.442324 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-737c4e90-6fde-46c5-a738-340d5b88bc0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baa7354ee46143dc846d99bcd0137cc3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:08.442324 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-737c4e90-6fde-46c5-a738-340d5b88bc0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baa7354ee46143dc846d99bcd0137cc3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:08.463539 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 886/1785] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:08 2022] DELETE /identity/v3/users/baa7354ee46143dc846d99bcd0137cc3 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:08.481114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d0ee9-b46e-4e2c-8b12-bbb370faf9e5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:08.547934 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:08.554906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f171d7b-2de0-48eb-a82e-a52c4c87db44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:08.556319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f171d7b-2de0-48eb-a82e-a52c4c87db44 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:08.559953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f171d7b-2de0-48eb-a82e-a52c4c87db44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:08.559953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f171d7b-2de0-48eb-a82e-a52c4c87db44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:08.559953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f171d7b-2de0-48eb-a82e-a52c4c87db44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:08.561101 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f171d7b-2de0-48eb-a82e-a52c4c87db44 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:08.579584 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 900/1786] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:08.581284 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c4e90-6fde-46c5-a738-340d5b88bc0f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:08.587295 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-737c4e90-6fde-46c5-a738-340d5b88bc0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:08.602882 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c4e90-6fde-46c5-a738-340d5b88bc0f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:08.611245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4611b-664f-4831-a0eb-e5a25bb32430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:08.612892 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4611b-664f-4831-a0eb-e5a25bb32430 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:08.613260 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4611b-664f-4831-a0eb-e5a25bb32430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:08.613798 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4611b-664f-4831-a0eb-e5a25bb32430 admin admin] PATH_INFO: `/v3/projects/433968dd9da34d889fbfaa517bf22e09/users/0cb606739bfb4c78aa9dac40cecc26ae/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:08.616020 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4611b-664f-4831-a0eb-e5a25bb32430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=433968dd9da34d889fbfaa517bf22e09, user_id=0cb606739bfb4c78aa9dac40cecc26ae, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:08.619832 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4611b-664f-4831-a0eb-e5a25bb32430 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:08.633629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 887/1787] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:08 2022] PUT /identity/v3/projects/433968dd9da34d889fbfaa517bf22e09/users/0cb606739bfb4c78aa9dac40cecc26ae/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:08.644559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5af77-c617-421c-b6dc-686f8a73b274 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:08.644854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5af77-c617-421c-b6dc-686f8a73b274 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:08.645141 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5af77-c617-421c-b6dc-686f8a73b274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:08.685160 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-76c5af77-c617-421c-b6dc-686f8a73b274 None None] MFA Rules not processed for user `0cb606739bfb4c78aa9dac40cecc26ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:08.686524 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-76c5af77-c617-421c-b6dc-686f8a73b274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:08.777568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 901/1788] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:08 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:08.789926 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4611b-664f-4831-a0eb-e5a25bb32430 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:08.791865 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b9d4611b-664f-4831-a0eb-e5a25bb32430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:08.840447 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:08.864211 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4611b-664f-4831-a0eb-e5a25bb32430 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:08.869258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4aa138-3572-4059-b98e-2749524bc562 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:08.869557 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4aa138-3572-4059-b98e-2749524bc562 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:08.870802 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4aa138-3572-4059-b98e-2749524bc562 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:08.870802 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4aa138-3572-4059-b98e-2749524bc562 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:08.882158 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4aa138-3572-4059-b98e-2749524bc562 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:08.883620 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4aa138-3572-4059-b98e-2749524bc562 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:08.896561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 888/1789] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:08 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:09.152030 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5af77-c617-421c-b6dc-686f8a73b274 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:09.167602 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5af77-c617-421c-b6dc-686f8a73b274 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:09.174496 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b2c92-df78-4bf6-95e1-133482a1be1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:09.175370 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b2c92-df78-4bf6-95e1-133482a1be1d service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:09.175763 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b2c92-df78-4bf6-95e1-133482a1be1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:09.176144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b2c92-df78-4bf6-95e1-133482a1be1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:09.178097 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2e2b2c92-df78-4bf6-95e1-133482a1be1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:09.266537 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b2c92-df78-4bf6-95e1-133482a1be1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:09.268305 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b2c92-df78-4bf6-95e1-133482a1be1d service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:09.283366 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 902/1790] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:09 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:09.693212 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4aa138-3572-4059-b98e-2749524bc562 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:09.694929 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8e4aa138-3572-4059-b98e-2749524bc562 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:09.804209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4aa138-3572-4059-b98e-2749524bc562 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:09.811739 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe8d31-f984-4029-82a3-453e962d7c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:09.812972 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe8d31-f984-4029-82a3-453e962d7c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:09.813334 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe8d31-f984-4029-82a3-453e962d7c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:09.813727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe8d31-f984-4029-82a3-453e962d7c94 admin admin] PATH_INFO: `/v3/users/58181dfcd0ff4abea8494655d837c4d8` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:09.815504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe8d31-f984-4029-82a3-453e962d7c94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58181dfcd0ff4abea8494655d837c4d8)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:09.818137 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe8d31-f984-4029-82a3-453e962d7c94 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:09.943447 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b2c92-df78-4bf6-95e1-133482a1be1d service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:09.947245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2e2b2c92-df78-4bf6-95e1-133482a1be1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:09.968183 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eabe8d31-f984-4029-82a3-453e962d7c94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58181dfcd0ff4abea8494655d837c4d8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:09.968585 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eabe8d31-f984-4029-82a3-453e962d7c94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58181dfcd0ff4abea8494655d837c4d8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:09.975572 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b2c92-df78-4bf6-95e1-133482a1be1d service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:09.981258 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec941f8-f023-40ff-aabd-d9e1729118a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:09.982524 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec941f8-f023-40ff-aabd-d9e1729118a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:09.983086 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec941f8-f023-40ff-aabd-d9e1729118a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:09.983435 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec941f8-f023-40ff-aabd-d9e1729118a6 admin admin] PATH_INFO: `/v3/projects/c14847b9cb394a778ff934ec6168dd8a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:09.984791 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec941f8-f023-40ff-aabd-d9e1729118a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c14847b9cb394a778ff934ec6168dd8a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:09.986947 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec941f8-f023-40ff-aabd-d9e1729118a6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:10.003885 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eabe8d31-f984-4029-82a3-453e962d7c94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58181dfcd0ff4abea8494655d837c4d8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:10.011979 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eabe8d31-f984-4029-82a3-453e962d7c94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58181dfcd0ff4abea8494655d837c4d8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:10.012671 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-eabe8d31-f984-4029-82a3-453e962d7c94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58181dfcd0ff4abea8494655d837c4d8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:10.030623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 889/1791] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:09 2022] DELETE /identity/v3/users/58181dfcd0ff4abea8494655d837c4d8 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:10.034796 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4178a7-f3a3-4d1e-b7bb-434f0dbaffb6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:10.035147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4178a7-f3a3-4d1e-b7bb-434f0dbaffb6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:10.035411 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4178a7-f3a3-4d1e-b7bb-434f0dbaffb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:10.042217 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6ec941f8-f023-40ff-aabd-d9e1729118a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c14847b9cb394a778ff934ec6168dd8a', 'request_id': 'req-6ec941f8-f023-40ff-aabd-d9e1729118a6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:10.050883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6ec941f8-f023-40ff-aabd-d9e1729118a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c14847b9cb394a778ff934ec6168dd8a', 'request_id': 'req-6ec941f8-f023-40ff-aabd-d9e1729118a6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:10.052853 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7f4178a7-f3a3-4d1e-b7bb-434f0dbaffb6 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:10.054088 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7f4178a7-f3a3-4d1e-b7bb-434f0dbaffb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:10.056554 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6ec941f8-f023-40ff-aabd-d9e1729118a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c14847b9cb394a778ff934ec6168dd8a', 'request_id': 'req-6ec941f8-f023-40ff-aabd-d9e1729118a6'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:10.067455 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:10.071771 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6ec941f8-f023-40ff-aabd-d9e1729118a6 admin admin] The token cache is being invalidate because project c14847b9cb394a778ff934ec6168dd8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:10.072351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6ec941f8-f023-40ff-aabd-d9e1729118a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:10.073571 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6ec941f8-f023-40ff-aabd-d9e1729118a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:10.077626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 903/1792] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:09 2022] DELETE /identity/v3/projects/c14847b9cb394a778ff934ec6168dd8a => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:10.083251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5edee50-6b68-49df-8e70-980e465b8007 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:10.083790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5edee50-6b68-49df-8e70-980e465b8007 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:10.084926 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5edee50-6b68-49df-8e70-980e465b8007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:10.111764 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b5edee50-6b68-49df-8e70-980e465b8007 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:10.113148 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b5edee50-6b68-49df-8e70-980e465b8007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:10.114763 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:10.127591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 890/1793] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:29:10.137440 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4178a7-f3a3-4d1e-b7bb-434f0dbaffb6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:10.138995 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7f4178a7-f3a3-4d1e-b7bb-434f0dbaffb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:10.176240 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 904/1794] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:29:10.183842 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23e910-5e2d-4735-8bde-d35cf6202c76 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:10.184136 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23e910-5e2d-4735-8bde-d35cf6202c76 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:10.184418 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23e910-5e2d-4735-8bde-d35cf6202c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:10.196284 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4178a7-f3a3-4d1e-b7bb-434f0dbaffb6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:10.199808 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36ad1d-e975-4461-9ef3-6c178b191fc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:10.200447 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36ad1d-e975-4461-9ef3-6c178b191fc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:10.200761 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36ad1d-e975-4461-9ef3-6c178b191fc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:10.201042 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36ad1d-e975-4461-9ef3-6c178b191fc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:10.202467 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7d23e910-5e2d-4735-8bde-d35cf6202c76 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:10.203685 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7d23e910-5e2d-4735-8bde-d35cf6202c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:10.210466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36ad1d-e975-4461-9ef3-6c178b191fc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:10.211709 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 905/1795] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:10.211897 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36ad1d-e975-4461-9ef3-6c178b191fc2 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:10.215847 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23e910-5e2d-4735-8bde-d35cf6202c76 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:10.223179 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 891/1796] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:29:10.225873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23e910-5e2d-4735-8bde-d35cf6202c76 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:10.231008 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80704dd9-7aeb-4803-a66c-7b43534fe67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:10.231663 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80704dd9-7aeb-4803-a66c-7b43534fe67f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:10.231945 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80704dd9-7aeb-4803-a66c-7b43534fe67f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:10.232972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80704dd9-7aeb-4803-a66c-7b43534fe67f admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:10.233971 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80704dd9-7aeb-4803-a66c-7b43534fe67f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:29:10.234326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80704dd9-7aeb-4803-a66c-7b43534fe67f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:10.235901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80704dd9-7aeb-4803-a66c-7b43534fe67f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:10.247818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 906/1797] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:29:10.252169 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36ad1d-e975-4461-9ef3-6c178b191fc2 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:10.261687 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36ad1d-e975-4461-9ef3-6c178b191fc2 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:10.266694 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff7b54-bed6-4baa-9359-6e60b482ffaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:10.267342 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff7b54-bed6-4baa-9359-6e60b482ffaa admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:10.267627 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff7b54-bed6-4baa-9359-6e60b482ffaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:10.267903 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff7b54-bed6-4baa-9359-6e60b482ffaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:10.268589 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff7b54-bed6-4baa-9359-6e60b482ffaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:10.270650 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff7b54-bed6-4baa-9359-6e60b482ffaa admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:10.299698 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 892/1798] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:10 2022] POST /identity/v3/projects => generated 355 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:10.303868 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80704dd9-7aeb-4803-a66c-7b43534fe67f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:10.316037 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80704dd9-7aeb-4803-a66c-7b43534fe67f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:10.391835 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802acd0f-fb84-481b-839f-d8f4764344a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:10.392115 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802acd0f-fb84-481b-839f-d8f4764344a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:10.392208 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802acd0f-fb84-481b-839f-d8f4764344a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:10.392426 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802acd0f-fb84-481b-839f-d8f4764344a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:10.393129 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802acd0f-fb84-481b-839f-d8f4764344a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:10.395262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802acd0f-fb84-481b-839f-d8f4764344a0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:10.425173 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 907/1799] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:10 2022] POST /identity/v3/users => generated 278 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:29:10.431363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff7b54-bed6-4baa-9359-6e60b482ffaa admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:10.455201 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff7b54-bed6-4baa-9359-6e60b482ffaa admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:10.460343 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458832ed-935a-4bd0-98fd-edd8b7e6ab97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:10.461059 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458832ed-935a-4bd0-98fd-edd8b7e6ab97 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:10.461420 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458832ed-935a-4bd0-98fd-edd8b7e6ab97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:10.461828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458832ed-935a-4bd0-98fd-edd8b7e6ab97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:10.462757 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458832ed-935a-4bd0-98fd-edd8b7e6ab97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:10.464596 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458832ed-935a-4bd0-98fd-edd8b7e6ab97 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:10.485623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 893/1800] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:10.488362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802acd0f-fb84-481b-839f-d8f4764344a0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:10.501837 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802acd0f-fb84-481b-839f-d8f4764344a0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:10.514611 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5eabde-94fd-4a5d-9d55-43c000e5acd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:10.515288 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5eabde-94fd-4a5d-9d55-43c000e5acd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:10.515570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5eabde-94fd-4a5d-9d55-43c000e5acd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:10.517229 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5eabde-94fd-4a5d-9d55-43c000e5acd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:10.518086 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5eabde-94fd-4a5d-9d55-43c000e5acd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:10.519735 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5eabde-94fd-4a5d-9d55-43c000e5acd1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:10.538472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 908/1801] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:10.546501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458832ed-935a-4bd0-98fd-edd8b7e6ab97 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:10.557785 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458832ed-935a-4bd0-98fd-edd8b7e6ab97 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:10.565110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5eabde-94fd-4a5d-9d55-43c000e5acd1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:10.566488 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91566f-1353-4141-8f94-7205a7b9da78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:10.566856 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ba5eabde-94fd-4a5d-9d55-43c000e5acd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:10.568009 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91566f-1353-4141-8f94-7205a7b9da78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:10.568744 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91566f-1353-4141-8f94-7205a7b9da78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:10.569245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91566f-1353-4141-8f94-7205a7b9da78 admin admin] PATH_INFO: `/v3/projects/93dbe78cd01b4b35a0f2677f0524f540/users/f5330763868b42a08e5016a56dcdd59b/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:10.591264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91566f-1353-4141-8f94-7205a7b9da78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93dbe78cd01b4b35a0f2677f0524f540, user_id=f5330763868b42a08e5016a56dcdd59b, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:10.594436 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5eabde-94fd-4a5d-9d55-43c000e5acd1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:10.596351 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91566f-1353-4141-8f94-7205a7b9da78 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:10.599208 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3d5f6-f569-4826-a31c-7562913d0289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:10.600071 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3d5f6-f569-4826-a31c-7562913d0289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:10.600357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3d5f6-f569-4826-a31c-7562913d0289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:10.600759 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3d5f6-f569-4826-a31c-7562913d0289 admin admin] PATH_INFO: `/v3/projects/0a6721a373b04c71bf1f725123d69891` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:10.602042 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3d5f6-f569-4826-a31c-7562913d0289 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a6721a373b04c71bf1f725123d69891)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:10.604330 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3d5f6-f569-4826-a31c-7562913d0289 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:10.611523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 894/1802] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:10 2022] PUT /identity/v3/projects/93dbe78cd01b4b35a0f2677f0524f540/users/f5330763868b42a08e5016a56dcdd59b/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:10.623560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222fcf5-f610-4b66-b5b3-ff9de45affff None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:10.623911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222fcf5-f610-4b66-b5b3-ff9de45affff None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:10.624327 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222fcf5-f610-4b66-b5b3-ff9de45affff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:10.651322 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0222fcf5-f610-4b66-b5b3-ff9de45affff None None] MFA Rules not processed for user `f5330763868b42a08e5016a56dcdd59b`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:10.652527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0222fcf5-f610-4b66-b5b3-ff9de45affff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:10.659643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-40a3d5f6-f569-4826-a31c-7562913d0289 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a6721a373b04c71bf1f725123d69891', 'request_id': 'req-40a3d5f6-f569-4826-a31c-7562913d0289'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:10.667300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-40a3d5f6-f569-4826-a31c-7562913d0289 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a6721a373b04c71bf1f725123d69891', 'request_id': 'req-40a3d5f6-f569-4826-a31c-7562913d0289'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:10.668680 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:10.674835 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-40a3d5f6-f569-4826-a31c-7562913d0289 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a6721a373b04c71bf1f725123d69891', 'request_id': 'req-40a3d5f6-f569-4826-a31c-7562913d0289'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:10.690910 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-40a3d5f6-f569-4826-a31c-7562913d0289 admin admin] The token cache is being invalidate because project 0a6721a373b04c71bf1f725123d69891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:10.691304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-40a3d5f6-f569-4826-a31c-7562913d0289 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:10.691965 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-40a3d5f6-f569-4826-a31c-7562913d0289 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:10.693820 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 909/1803] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:10 2022] DELETE /identity/v3/projects/0a6721a373b04c71bf1f725123d69891 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:10.697501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3d5f6-f569-4826-a31c-7562913d0289 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:10.699247 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-40a3d5f6-f569-4826-a31c-7562913d0289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:10.723523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 895/1804] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:10 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:10.727792 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222fcf5-f610-4b66-b5b3-ff9de45affff None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:10.729242 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0222fcf5-f610-4b66-b5b3-ff9de45affff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:10.768225 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:10.771314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3d5f6-f569-4826-a31c-7562913d0289 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:10.776605 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6fa0d-edbe-4277-af65-a0639c1304e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:10.777508 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6fa0d-edbe-4277-af65-a0639c1304e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:10.777824 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6fa0d-edbe-4277-af65-a0639c1304e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:10.778098 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6fa0d-edbe-4277-af65-a0639c1304e9 admin admin] PATH_INFO: `/v3/users/659d03f3fe734f6985a6a84feba5b272` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:10.780602 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f6fa0d-edbe-4277-af65-a0639c1304e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=659d03f3fe734f6985a6a84feba5b272)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:10.782689 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f6fa0d-edbe-4277-af65-a0639c1304e9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:10.801143 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222fcf5-f610-4b66-b5b3-ff9de45affff None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:10.806983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901ee0b-31d1-4947-9562-bb87f623b899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:10.808499 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901ee0b-31d1-4947-9562-bb87f623b899 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:10.809253 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901ee0b-31d1-4947-9562-bb87f623b899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:10.809710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901ee0b-31d1-4947-9562-bb87f623b899 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:10.810909 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6901ee0b-31d1-4947-9562-bb87f623b899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:10.812966 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6901ee0b-31d1-4947-9562-bb87f623b899 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:10.845936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 896/1805] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:10 2022] POST /identity/v3/projects => generated 355 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:29:10.846688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e7f6fa0d-edbe-4277-af65-a0639c1304e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '659d03f3fe734f6985a6a84feba5b272'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:10.859014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901ee0b-31d1-4947-9562-bb87f623b899 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:10.874453 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e7f6fa0d-edbe-4277-af65-a0639c1304e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '659d03f3fe734f6985a6a84feba5b272'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:10.875214 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e7f6fa0d-edbe-4277-af65-a0639c1304e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '659d03f3fe734f6985a6a84feba5b272'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:10.884281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e7f6fa0d-edbe-4277-af65-a0639c1304e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '659d03f3fe734f6985a6a84feba5b272'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:10.885227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e7f6fa0d-edbe-4277-af65-a0639c1304e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '659d03f3fe734f6985a6a84feba5b272'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:10.916858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 910/1806] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:10 2022] DELETE /identity/v3/users/659d03f3fe734f6985a6a84feba5b272 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:10.953005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901ee0b-31d1-4947-9562-bb87f623b899 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:11.043826 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6537a-e6b4-433d-b935-bbf79a5d3ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:11.045467 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6537a-e6b4-433d-b935-bbf79a5d3ebb admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:11.045957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6537a-e6b4-433d-b935-bbf79a5d3ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:11.046775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6537a-e6b4-433d-b935-bbf79a5d3ebb admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:11.047849 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6537a-e6b4-433d-b935-bbf79a5d3ebb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:11.050215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6537a-e6b4-433d-b935-bbf79a5d3ebb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:11.088556 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 897/1807] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:10 2022] POST /identity/v3/users => generated 284 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:29:11.093832 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6fa0d-edbe-4277-af65-a0639c1304e9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:11.097360 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e7f6fa0d-edbe-4277-af65-a0639c1304e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:11.115673 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6fa0d-edbe-4277-af65-a0639c1304e9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:11.131149 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f441908-9bbd-40ee-9afd-e830ffa13602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:11.132412 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f441908-9bbd-40ee-9afd-e830ffa13602 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:11.132704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f441908-9bbd-40ee-9afd-e830ffa13602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:11.133018 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f441908-9bbd-40ee-9afd-e830ffa13602 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:11.133929 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f441908-9bbd-40ee-9afd-e830ffa13602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:11.135534 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f441908-9bbd-40ee-9afd-e830ffa13602 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:11.153538 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 911/1808] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:11.153739 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6537a-e6b4-433d-b935-bbf79a5d3ebb admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:11.163834 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6537a-e6b4-433d-b935-bbf79a5d3ebb admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:11.169337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3b588-3699-4725-b561-cfbd77757cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:11.171034 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3b588-3699-4725-b561-cfbd77757cd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:11.171423 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3b588-3699-4725-b561-cfbd77757cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:11.171939 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3b588-3699-4725-b561-cfbd77757cd9 admin admin] PATH_INFO: `/v3/projects/387bb82a80fc434eb46a424c7c544762/users/8b85b9be871245f08ec3dbab4c9befe1/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:11.189816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f3b588-3699-4725-b561-cfbd77757cd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=387bb82a80fc434eb46a424c7c544762, user_id=8b85b9be871245f08ec3dbab4c9befe1, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:11.192613 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f3b588-3699-4725-b561-cfbd77757cd9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:11.208996 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 898/1809] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:11 2022] PUT /identity/v3/projects/387bb82a80fc434eb46a424c7c544762/users/8b85b9be871245f08ec3dbab4c9befe1/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:11.212956 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f441908-9bbd-40ee-9afd-e830ffa13602 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:11.226081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f441908-9bbd-40ee-9afd-e830ffa13602 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:11.293557 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:11.313424 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0de28-5a53-4623-8724-1cc4d1557d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:11.314181 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0de28-5a53-4623-8724-1cc4d1557d8c admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:11.314478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0de28-5a53-4623-8724-1cc4d1557d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:11.314764 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0de28-5a53-4623-8724-1cc4d1557d8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:11.315493 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0de28-5a53-4623-8724-1cc4d1557d8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:11.317272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0de28-5a53-4623-8724-1cc4d1557d8c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:11.332671 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 912/1810] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:11.337023 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3b588-3699-4725-b561-cfbd77757cd9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:11.352929 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3b588-3699-4725-b561-cfbd77757cd9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:11.359408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92e27b-be7b-42f7-b1cd-7ed39c58eca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:11.360478 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92e27b-be7b-42f7-b1cd-7ed39c58eca9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:11.360851 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92e27b-be7b-42f7-b1cd-7ed39c58eca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:11.361187 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92e27b-be7b-42f7-b1cd-7ed39c58eca9 admin admin] PATH_INFO: `/v3/projects/387bb82a80fc434eb46a424c7c544762/users/8b85b9be871245f08ec3dbab4c9befe1/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:11.363452 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92e27b-be7b-42f7-b1cd-7ed39c58eca9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=387bb82a80fc434eb46a424c7c544762, user_id=8b85b9be871245f08ec3dbab4c9befe1, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:11.367323 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92e27b-be7b-42f7-b1cd-7ed39c58eca9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:11.394736 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 899/1811] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:11 2022] PUT /identity/v3/projects/387bb82a80fc434eb46a424c7c544762/users/8b85b9be871245f08ec3dbab4c9befe1/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:11.405079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b374cb-75a4-4887-9b72-e152358a29a0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:11.405405 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b374cb-75a4-4887-9b72-e152358a29a0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:11.405731 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b374cb-75a4-4887-9b72-e152358a29a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:11.434901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-99b374cb-75a4-4887-9b72-e152358a29a0 None None] MFA Rules not processed for user `8b85b9be871245f08ec3dbab4c9befe1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:11.437136 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-99b374cb-75a4-4887-9b72-e152358a29a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:11.476114 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92e27b-be7b-42f7-b1cd-7ed39c58eca9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:11.478122 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1e92e27b-be7b-42f7-b1cd-7ed39c58eca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:11.518556 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:11.537626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 913/1812] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:11 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:11.555281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b374cb-75a4-4887-9b72-e152358a29a0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:11.556655 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-99b374cb-75a4-4887-9b72-e152358a29a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:11.595367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92e27b-be7b-42f7-b1cd-7ed39c58eca9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:11.600112 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a016806-56ea-4a82-a3df-9e53215b3651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:11.601037 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a016806-56ea-4a82-a3df-9e53215b3651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:11.601476 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a016806-56ea-4a82-a3df-9e53215b3651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:11.601777 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a016806-56ea-4a82-a3df-9e53215b3651 admin admin] PATH_INFO: `/v3/projects/4a572f8d3641497f9f9be8866b5c89a1` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:11.603101 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a016806-56ea-4a82-a3df-9e53215b3651 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a572f8d3641497f9f9be8866b5c89a1)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:11.606008 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a016806-56ea-4a82-a3df-9e53215b3651 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:11.624093 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b374cb-75a4-4887-9b72-e152358a29a0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:11.628424 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ba825-3010-4146-b648-e602c38d1899 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:11.629075 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ba825-3010-4146-b648-e602c38d1899 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:11.629361 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ba825-3010-4146-b648-e602c38d1899 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:11.629677 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ba825-3010-4146-b648-e602c38d1899 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:11.640731 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ba825-3010-4146-b648-e602c38d1899 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:11.641725 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ba825-3010-4146-b648-e602c38d1899 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:11.653231 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 914/1813] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:11 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:11.664123 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9a016806-56ea-4a82-a3df-9e53215b3651 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a572f8d3641497f9f9be8866b5c89a1', 'request_id': 'req-9a016806-56ea-4a82-a3df-9e53215b3651'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:11.671255 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9a016806-56ea-4a82-a3df-9e53215b3651 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a572f8d3641497f9f9be8866b5c89a1', 'request_id': 'req-9a016806-56ea-4a82-a3df-9e53215b3651'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:11.678691 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9a016806-56ea-4a82-a3df-9e53215b3651 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a572f8d3641497f9f9be8866b5c89a1', 'request_id': 'req-9a016806-56ea-4a82-a3df-9e53215b3651'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:11.694969 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9a016806-56ea-4a82-a3df-9e53215b3651 admin admin] The token cache is being invalidate because project 4a572f8d3641497f9f9be8866b5c89a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:11.695261 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9a016806-56ea-4a82-a3df-9e53215b3651 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:11.695990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9a016806-56ea-4a82-a3df-9e53215b3651 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:11.700739 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 900/1814] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:11 2022] DELETE /identity/v3/projects/4a572f8d3641497f9f9be8866b5c89a1 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:11.703279 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf5b16-e11c-42b4-bff2-5d8cce1daf66 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:11.703837 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf5b16-e11c-42b4-bff2-5d8cce1daf66 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:11.704185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf5b16-e11c-42b4-bff2-5d8cce1daf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:11.728338 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-41bf5b16-e11c-42b4-bff2-5d8cce1daf66 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:11.729618 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-41bf5b16-e11c-42b4-bff2-5d8cce1daf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:11.738591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a016806-56ea-4a82-a3df-9e53215b3651 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:11.740482 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9a016806-56ea-4a82-a3df-9e53215b3651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:11.801588 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 915/1815] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:11.803836 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a016806-56ea-4a82-a3df-9e53215b3651 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:11.808493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a239d-a1ff-48cd-8bcf-faac42ff255d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:11.809148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a239d-a1ff-48cd-8bcf-faac42ff255d service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:11.809435 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a239d-a1ff-48cd-8bcf-faac42ff255d service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:11.809750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a239d-a1ff-48cd-8bcf-faac42ff255d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:11.810791 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224843a-8763-4ecd-9acd-1b74ff03f6a1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:11.811084 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224843a-8763-4ecd-9acd-1b74ff03f6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:11.811374 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224843a-8763-4ecd-9acd-1b74ff03f6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:11.812915 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e04a239d-a1ff-48cd-8bcf-faac42ff255d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:11.829833 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5224843a-8763-4ecd-9acd-1b74ff03f6a1 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:11.831016 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5224843a-8763-4ecd-9acd-1b74ff03f6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:11.839134 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 916/1816] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:29:11.843502 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224843a-8763-4ecd-9acd-1b74ff03f6a1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:11.852208 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224843a-8763-4ecd-9acd-1b74ff03f6a1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:11.856721 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a6d9a-5fc5-4710-be1b-4586fe832547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:11.857965 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a6d9a-5fc5-4710-be1b-4586fe832547 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:11.858253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a6d9a-5fc5-4710-be1b-4586fe832547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:11.858534 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a6d9a-5fc5-4710-be1b-4586fe832547 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:11.859255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a6d9a-5fc5-4710-be1b-4586fe832547 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:29:11.859554 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a6d9a-5fc5-4710-be1b-4586fe832547 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:11.861155 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a6d9a-5fc5-4710-be1b-4586fe832547 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:11.863596 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:11.875986 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 917/1817] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:29:11.877892 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a239d-a1ff-48cd-8bcf-faac42ff255d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:11.879365 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a239d-a1ff-48cd-8bcf-faac42ff255d service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:11.880140 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a6d9a-5fc5-4710-be1b-4586fe832547 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:11.888826 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 901/1818] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:11 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:29:11.890763 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a6d9a-5fc5-4710-be1b-4586fe832547 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:11.896513 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e08ea-ce25-4e9a-bcb6-c08c84a00083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:11.897164 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e08ea-ce25-4e9a-bcb6-c08c84a00083 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:11.897451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e08ea-ce25-4e9a-bcb6-c08c84a00083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:11.897761 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e08ea-ce25-4e9a-bcb6-c08c84a00083 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:11.898454 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e08ea-ce25-4e9a-bcb6-c08c84a00083 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:11.900403 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e08ea-ce25-4e9a-bcb6-c08c84a00083 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:11.931068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 918/1819] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:11 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:29:11.935493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a239d-a1ff-48cd-8bcf-faac42ff255d service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:11.942046 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:11.948421 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a239d-a1ff-48cd-8bcf-faac42ff255d service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:12.023671 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55feb74-cdc5-45c5-a1aa-6757ef2e2110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:12.024448 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55feb74-cdc5-45c5-a1aa-6757ef2e2110 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:12.024863 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55feb74-cdc5-45c5-a1aa-6757ef2e2110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:12.029640 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55feb74-cdc5-45c5-a1aa-6757ef2e2110 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:12.029640 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55feb74-cdc5-45c5-a1aa-6757ef2e2110 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:12.029640 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55feb74-cdc5-45c5-a1aa-6757ef2e2110 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:12.052552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 902/1820] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:11 2022] POST /identity/v3/users => generated 285 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:29:12.056537 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e08ea-ce25-4e9a-bcb6-c08c84a00083 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:12.067730 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e08ea-ce25-4e9a-bcb6-c08c84a00083 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:12.072486 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6691632-df96-4fdd-9897-0254c789a5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:12.073146 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6691632-df96-4fdd-9897-0254c789a5cd admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:12.073433 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6691632-df96-4fdd-9897-0254c789a5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:12.073754 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6691632-df96-4fdd-9897-0254c789a5cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:12.074480 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6691632-df96-4fdd-9897-0254c789a5cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:12.076134 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6691632-df96-4fdd-9897-0254c789a5cd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:12.086423 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 919/1821] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:12.090349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55feb74-cdc5-45c5-a1aa-6757ef2e2110 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:12.100541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55feb74-cdc5-45c5-a1aa-6757ef2e2110 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:12.105245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de69183-ec5c-41a9-a646-d5be9e010a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:12.105938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de69183-ec5c-41a9-a646-d5be9e010a0b admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:12.106248 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de69183-ec5c-41a9-a646-d5be9e010a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:12.106526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de69183-ec5c-41a9-a646-d5be9e010a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:12.107816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de69183-ec5c-41a9-a646-d5be9e010a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:12.109460 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de69183-ec5c-41a9-a646-d5be9e010a0b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:12.119416 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 903/1822] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:12.123886 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6691632-df96-4fdd-9897-0254c789a5cd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:12.134056 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6691632-df96-4fdd-9897-0254c789a5cd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:12.139249 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a557a09-deb0-49db-acb3-52b03d39d79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:12.140218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a557a09-deb0-49db-acb3-52b03d39d79a admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:12.140508 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a557a09-deb0-49db-acb3-52b03d39d79a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:12.140783 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a557a09-deb0-49db-acb3-52b03d39d79a admin admin] PATH_INFO: `/v3/projects/dfd4f716b10b4cb88c8f25c725b98944/users/749af856c6cf4d4d8fe34250418eda43/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:12.156205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a557a09-deb0-49db-acb3-52b03d39d79a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfd4f716b10b4cb88c8f25c725b98944, user_id=749af856c6cf4d4d8fe34250418eda43, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:12.159244 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a557a09-deb0-49db-acb3-52b03d39d79a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:12.172971 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 920/1823] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:12 2022] PUT /identity/v3/projects/dfd4f716b10b4cb88c8f25c725b98944/users/749af856c6cf4d4d8fe34250418eda43/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:12.183133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384729a-b8d8-4aed-ac16-f14a3e4629fb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:12.183428 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384729a-b8d8-4aed-ac16-f14a3e4629fb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:12.183707 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384729a-b8d8-4aed-ac16-f14a3e4629fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:12.210540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9384729a-b8d8-4aed-ac16-f14a3e4629fb None None] MFA Rules not processed for user `749af856c6cf4d4d8fe34250418eda43`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:12.211732 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9384729a-b8d8-4aed-ac16-f14a3e4629fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:12.277460 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 904/1824] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:12 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:12.282526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a557a09-deb0-49db-acb3-52b03d39d79a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:12.356155 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a557a09-deb0-49db-acb3-52b03d39d79a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:12.361457 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36738f3d-b7e1-4958-bac5-d4b675b735e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:12.362153 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36738f3d-b7e1-4958-bac5-d4b675b735e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:12.362438 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36738f3d-b7e1-4958-bac5-d4b675b735e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:12.362718 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36738f3d-b7e1-4958-bac5-d4b675b735e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:12.363415 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36738f3d-b7e1-4958-bac5-d4b675b735e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:12.365352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36738f3d-b7e1-4958-bac5-d4b675b735e3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:12.396013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 921/1825] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:12 2022] POST /identity/v3/projects => generated 369 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:29:12.397621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384729a-b8d8-4aed-ac16-f14a3e4629fb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:12.409091 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384729a-b8d8-4aed-ac16-f14a3e4629fb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:12.481933 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc15a9-4eb6-4721-81b6-a000847bc233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:12.483422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc15a9-4eb6-4721-81b6-a000847bc233 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:12.483548 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc15a9-4eb6-4721-81b6-a000847bc233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:12.483796 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc15a9-4eb6-4721-81b6-a000847bc233 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:12.484492 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc15a9-4eb6-4721-81b6-a000847bc233 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:12.486507 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc15a9-4eb6-4721-81b6-a000847bc233 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:12.506594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:12.512074 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 905/1826] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:12 2022] POST /identity/v3/users => generated 291 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:29:12.516402 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36738f3d-b7e1-4958-bac5-d4b675b735e3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:12.526624 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36738f3d-b7e1-4958-bac5-d4b675b735e3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:12.531923 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed589826-54b0-48f5-b06f-f347c993741f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:12.532614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed589826-54b0-48f5-b06f-f347c993741f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:12.532970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed589826-54b0-48f5-b06f-f347c993741f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:12.533256 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed589826-54b0-48f5-b06f-f347c993741f admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:12.534284 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed589826-54b0-48f5-b06f-f347c993741f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:12.535836 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed589826-54b0-48f5-b06f-f347c993741f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:12.547271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 922/1827] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:12.551495 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc15a9-4eb6-4721-81b6-a000847bc233 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:12.561653 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc15a9-4eb6-4721-81b6-a000847bc233 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:12.566485 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554dc7bf-0ced-4a79-ad03-c7f924c77f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:12.567352 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554dc7bf-0ced-4a79-ad03-c7f924c77f4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:12.567637 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554dc7bf-0ced-4a79-ad03-c7f924c77f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:12.567920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554dc7bf-0ced-4a79-ad03-c7f924c77f4f admin admin] PATH_INFO: `/v3/projects/3c541da604f842a2be9839c037801f67/users/7d2111f37e724f2a998566fd5d47a65d/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:12.582203 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554dc7bf-0ced-4a79-ad03-c7f924c77f4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c541da604f842a2be9839c037801f67, user_id=7d2111f37e724f2a998566fd5d47a65d, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:12.584904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554dc7bf-0ced-4a79-ad03-c7f924c77f4f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:12.599019 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 906/1828] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:12 2022] PUT /identity/v3/projects/3c541da604f842a2be9839c037801f67/users/7d2111f37e724f2a998566fd5d47a65d/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:12.602838 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed589826-54b0-48f5-b06f-f347c993741f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:12.614139 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed589826-54b0-48f5-b06f-f347c993741f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:12.669928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3992e2f-cedc-4ed3-a4fc-5bc2a7aac25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:12.670658 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3992e2f-cedc-4ed3-a4fc-5bc2a7aac25d admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:12.670946 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3992e2f-cedc-4ed3-a4fc-5bc2a7aac25d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:12.671223 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3992e2f-cedc-4ed3-a4fc-5bc2a7aac25d admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:12.671936 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3992e2f-cedc-4ed3-a4fc-5bc2a7aac25d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:12.673524 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3992e2f-cedc-4ed3-a4fc-5bc2a7aac25d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:12.682988 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 923/1829] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:12.687553 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554dc7bf-0ced-4a79-ad03-c7f924c77f4f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:12.697910 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554dc7bf-0ced-4a79-ad03-c7f924c77f4f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:12.704274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0b052-a29a-4c5c-86bd-d7926123d890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:12.705291 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0b052-a29a-4c5c-86bd-d7926123d890 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:12.705972 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0b052-a29a-4c5c-86bd-d7926123d890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:12.705972 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0b052-a29a-4c5c-86bd-d7926123d890 admin admin] PATH_INFO: `/v3/projects/3c541da604f842a2be9839c037801f67/users/7d2111f37e724f2a998566fd5d47a65d/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:12.707943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d0b052-a29a-4c5c-86bd-d7926123d890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c541da604f842a2be9839c037801f67, user_id=7d2111f37e724f2a998566fd5d47a65d, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:12.710662 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d0b052-a29a-4c5c-86bd-d7926123d890 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:12.722198 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 907/1830] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:12 2022] PUT /identity/v3/projects/3c541da604f842a2be9839c037801f67/users/7d2111f37e724f2a998566fd5d47a65d/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:12.732980 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd50eff-0f47-4a3f-a57b-fc00671ead44 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:12.733304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd50eff-0f47-4a3f-a57b-fc00671ead44 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:12.733615 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd50eff-0f47-4a3f-a57b-fc00671ead44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:12.751543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:12.765125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-bfd50eff-0f47-4a3f-a57b-fc00671ead44 None None] MFA Rules not processed for user `7d2111f37e724f2a998566fd5d47a65d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:12.766411 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bfd50eff-0f47-4a3f-a57b-fc00671ead44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:12.845601 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 924/1831] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:12 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:12.856441 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0b052-a29a-4c5c-86bd-d7926123d890 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:12.858930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f2d0b052-a29a-4c5c-86bd-d7926123d890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:12.916747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0b052-a29a-4c5c-86bd-d7926123d890 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:12.921063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca236e1-6a24-48b8-af62-be71c0d66fef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:12.921764 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca236e1-6a24-48b8-af62-be71c0d66fef service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:12.922052 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca236e1-6a24-48b8-af62-be71c0d66fef service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:12.922332 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca236e1-6a24-48b8-af62-be71c0d66fef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:12.931262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca236e1-6a24-48b8-af62-be71c0d66fef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:12.932684 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca236e1-6a24-48b8-af62-be71c0d66fef service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:12.942321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 908/1832] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:12 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:14.333691 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd50eff-0f47-4a3f-a57b-fc00671ead44 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:14.395480 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd50eff-0f47-4a3f-a57b-fc00671ead44 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:14.399984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2352d-ab42-4595-bb3b-6ab072cac459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:14.400843 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2352d-ab42-4595-bb3b-6ab072cac459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:14.401130 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2352d-ab42-4595-bb3b-6ab072cac459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:14.401410 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2352d-ab42-4595-bb3b-6ab072cac459 admin admin] PATH_INFO: `/v3/users/749af856c6cf4d4d8fe34250418eda43` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:14.402612 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f2352d-ab42-4595-bb3b-6ab072cac459 admin admin] RBAC: Authorizing `identity:delete_user(user_id=749af856c6cf4d4d8fe34250418eda43)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:14.404546 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f2352d-ab42-4595-bb3b-6ab072cac459 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:14.449385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b4f2352d-ab42-4595-bb3b-6ab072cac459 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '749af856c6cf4d4d8fe34250418eda43'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:14.461252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b4f2352d-ab42-4595-bb3b-6ab072cac459 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '749af856c6cf4d4d8fe34250418eda43'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:14.461959 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b4f2352d-ab42-4595-bb3b-6ab072cac459 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '749af856c6cf4d4d8fe34250418eda43'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:14.469205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b4f2352d-ab42-4595-bb3b-6ab072cac459 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '749af856c6cf4d4d8fe34250418eda43'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:14.469887 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b4f2352d-ab42-4595-bb3b-6ab072cac459 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '749af856c6cf4d4d8fe34250418eda43'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:14.486459 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 925/1833] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:14 2022] DELETE /identity/v3/users/749af856c6cf4d4d8fe34250418eda43 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:14.491641 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c23ac-881c-4d7b-b289-c0071ccaf92c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:14.492158 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c23ac-881c-4d7b-b289-c0071ccaf92c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:14.492158 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c23ac-881c-4d7b-b289-c0071ccaf92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:14.510241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ed6c23ac-881c-4d7b-b289-c0071ccaf92c None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:14.511421 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ed6c23ac-881c-4d7b-b289-c0071ccaf92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:14.573349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 909/1834] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:14.582792 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2352d-ab42-4595-bb3b-6ab072cac459 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:14.584222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b4f2352d-ab42-4595-bb3b-6ab072cac459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:14.599535 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:14.644221 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2352d-ab42-4595-bb3b-6ab072cac459 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:14.647831 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589a3d2-667e-4549-ad20-eca8d2c4de70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:14.648473 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589a3d2-667e-4549-ad20-eca8d2c4de70 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:14.648760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589a3d2-667e-4549-ad20-eca8d2c4de70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:14.649042 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589a3d2-667e-4549-ad20-eca8d2c4de70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:14.658488 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4589a3d2-667e-4549-ad20-eca8d2c4de70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:14.659923 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4589a3d2-667e-4549-ad20-eca8d2c4de70 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:14.669187 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 926/1835] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:29:14.973377 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c23ac-881c-4d7b-b289-c0071ccaf92c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:14.975967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ed6c23ac-881c-4d7b-b289-c0071ccaf92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:14.988906 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c23ac-881c-4d7b-b289-c0071ccaf92c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:14.993459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e575cef-b79c-400b-8319-7beb900531d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:14.994380 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e575cef-b79c-400b-8319-7beb900531d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:14.994667 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e575cef-b79c-400b-8319-7beb900531d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:14.994953 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e575cef-b79c-400b-8319-7beb900531d4 admin admin] PATH_INFO: `/v3/projects/dfd4f716b10b4cb88c8f25c725b98944` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:14.996180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e575cef-b79c-400b-8319-7beb900531d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfd4f716b10b4cb88c8f25c725b98944)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:14.998256 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e575cef-b79c-400b-8319-7beb900531d4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:15.046784 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3e575cef-b79c-400b-8319-7beb900531d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfd4f716b10b4cb88c8f25c725b98944', 'request_id': 'req-3e575cef-b79c-400b-8319-7beb900531d4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:15.052775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3e575cef-b79c-400b-8319-7beb900531d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfd4f716b10b4cb88c8f25c725b98944', 'request_id': 'req-3e575cef-b79c-400b-8319-7beb900531d4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:15.057319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:15.058654 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3e575cef-b79c-400b-8319-7beb900531d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfd4f716b10b4cb88c8f25c725b98944', 'request_id': 'req-3e575cef-b79c-400b-8319-7beb900531d4'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:15.078038 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3e575cef-b79c-400b-8319-7beb900531d4 admin admin] The token cache is being invalidate because project dfd4f716b10b4cb88c8f25c725b98944 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:15.078255 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3e575cef-b79c-400b-8319-7beb900531d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:15.078871 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3e575cef-b79c-400b-8319-7beb900531d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:15.080409 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 910/1836] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:14 2022] DELETE /identity/v3/projects/dfd4f716b10b4cb88c8f25c725b98944 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:15.084338 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589a3d2-667e-4549-ad20-eca8d2c4de70 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:15.094884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589a3d2-667e-4549-ad20-eca8d2c4de70 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:15.186664 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44381e95-97d3-4667-a7c7-c36380412801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:15.187815 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44381e95-97d3-4667-a7c7-c36380412801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:15.188106 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44381e95-97d3-4667-a7c7-c36380412801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:15.188389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44381e95-97d3-4667-a7c7-c36380412801 admin admin] PATH_INFO: `/v3/users/7d2111f37e724f2a998566fd5d47a65d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:15.190527 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44381e95-97d3-4667-a7c7-c36380412801 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d2111f37e724f2a998566fd5d47a65d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:15.193457 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44381e95-97d3-4667-a7c7-c36380412801 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:15.292921 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-44381e95-97d3-4667-a7c7-c36380412801 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d2111f37e724f2a998566fd5d47a65d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:15.321101 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-44381e95-97d3-4667-a7c7-c36380412801 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d2111f37e724f2a998566fd5d47a65d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:15.321891 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-44381e95-97d3-4667-a7c7-c36380412801 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d2111f37e724f2a998566fd5d47a65d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:15.334743 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-44381e95-97d3-4667-a7c7-c36380412801 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d2111f37e724f2a998566fd5d47a65d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:15.335700 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-44381e95-97d3-4667-a7c7-c36380412801 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d2111f37e724f2a998566fd5d47a65d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:15.356165 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 927/1837] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:15 2022] DELETE /identity/v3/users/7d2111f37e724f2a998566fd5d47a65d => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:15.635145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e575cef-b79c-400b-8319-7beb900531d4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:15.637286 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3e575cef-b79c-400b-8319-7beb900531d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:15.653903 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e575cef-b79c-400b-8319-7beb900531d4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:15.711856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f9aaf-7b3e-4cfe-88c7-05984bee176e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:15.712715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f9aaf-7b3e-4cfe-88c7-05984bee176e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:15.712996 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f9aaf-7b3e-4cfe-88c7-05984bee176e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:15.713272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f9aaf-7b3e-4cfe-88c7-05984bee176e admin admin] PATH_INFO: `/v3/projects/3c541da604f842a2be9839c037801f67` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:15.714474 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f9aaf-7b3e-4cfe-88c7-05984bee176e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c541da604f842a2be9839c037801f67)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:15.716394 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f9aaf-7b3e-4cfe-88c7-05984bee176e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:15.760363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2a4f9aaf-7b3e-4cfe-88c7-05984bee176e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c541da604f842a2be9839c037801f67', 'request_id': 'req-2a4f9aaf-7b3e-4cfe-88c7-05984bee176e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:15.766477 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2a4f9aaf-7b3e-4cfe-88c7-05984bee176e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c541da604f842a2be9839c037801f67', 'request_id': 'req-2a4f9aaf-7b3e-4cfe-88c7-05984bee176e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:15.771860 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2a4f9aaf-7b3e-4cfe-88c7-05984bee176e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c541da604f842a2be9839c037801f67', 'request_id': 'req-2a4f9aaf-7b3e-4cfe-88c7-05984bee176e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:15.785799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2a4f9aaf-7b3e-4cfe-88c7-05984bee176e admin admin] The token cache is being invalidate because project 3c541da604f842a2be9839c037801f67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:15.786095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2a4f9aaf-7b3e-4cfe-88c7-05984bee176e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:15.786788 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-2a4f9aaf-7b3e-4cfe-88c7-05984bee176e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:15.788351 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 911/1838] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:15 2022] DELETE /identity/v3/projects/3c541da604f842a2be9839c037801f67 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:15.793182 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da751bd-7840-40fb-8daa-dbc6c66fd08c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:15.793478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da751bd-7840-40fb-8daa-dbc6c66fd08c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:15.793790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da751bd-7840-40fb-8daa-dbc6c66fd08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:15.814977 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5da751bd-7840-40fb-8daa-dbc6c66fd08c None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:15.816308 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5da751bd-7840-40fb-8daa-dbc6c66fd08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:15.846180 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:15.877948 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 928/1839] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:15.885789 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f2058-5960-4a55-b24e-587ce12d7f3d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:15.886082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f2058-5960-4a55-b24e-587ce12d7f3d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:15.886364 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f2058-5960-4a55-b24e-587ce12d7f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:15.905792 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-333f2058-5960-4a55-b24e-587ce12d7f3d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:15.906989 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-333f2058-5960-4a55-b24e-587ce12d7f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:15.914849 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 912/1840] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:15.919088 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da751bd-7840-40fb-8daa-dbc6c66fd08c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:15.928901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da751bd-7840-40fb-8daa-dbc6c66fd08c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:15.933443 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc66b5-b90f-4553-b02c-30d7c3e5ca6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:15.934125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc66b5-b90f-4553-b02c-30d7c3e5ca6b admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:15.934415 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc66b5-b90f-4553-b02c-30d7c3e5ca6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:15.934694 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc66b5-b90f-4553-b02c-30d7c3e5ca6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:15.935422 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc66b5-b90f-4553-b02c-30d7c3e5ca6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:29:15.935887 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc66b5-b90f-4553-b02c-30d7c3e5ca6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:15.937784 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc66b5-b90f-4553-b02c-30d7c3e5ca6b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:15.950201 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 929/1841] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:29:15.955285 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f2058-5960-4a55-b24e-587ce12d7f3d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:15.964610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f2058-5960-4a55-b24e-587ce12d7f3d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:15.970694 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c7c72-6b8b-44f4-bbfc-dd498cb05e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:15.971368 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c7c72-6b8b-44f4-bbfc-dd498cb05e19 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:15.971649 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c7c72-6b8b-44f4-bbfc-dd498cb05e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:15.971923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c7c72-6b8b-44f4-bbfc-dd498cb05e19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:15.972613 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c7c72-6b8b-44f4-bbfc-dd498cb05e19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:15.974544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c7c72-6b8b-44f4-bbfc-dd498cb05e19 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:16.006933 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 913/1842] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:15 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:16.008107 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc66b5-b90f-4553-b02c-30d7c3e5ca6b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:16.022019 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc66b5-b90f-4553-b02c-30d7c3e5ca6b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:16.087363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db66a17-d011-4814-aa4a-aba5013844dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:16.088095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db66a17-d011-4814-aa4a-aba5013844dc admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:16.088387 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db66a17-d011-4814-aa4a-aba5013844dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:16.088666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db66a17-d011-4814-aa4a-aba5013844dc admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:16.089352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db66a17-d011-4814-aa4a-aba5013844dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:16.091318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db66a17-d011-4814-aa4a-aba5013844dc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:16.118317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 930/1843] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:16 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:29:16.122515 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c7c72-6b8b-44f4-bbfc-dd498cb05e19 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:16.134164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c7c72-6b8b-44f4-bbfc-dd498cb05e19 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:16.139460 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6030bf4-b93a-452b-8522-6d52a155c50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:16.140175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6030bf4-b93a-452b-8522-6d52a155c50e admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:16.140529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6030bf4-b93a-452b-8522-6d52a155c50e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:16.140877 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6030bf4-b93a-452b-8522-6d52a155c50e admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:16.141638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6030bf4-b93a-452b-8522-6d52a155c50e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:16.143259 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6030bf4-b93a-452b-8522-6d52a155c50e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:16.157663 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 914/1844] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:16.159054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db66a17-d011-4814-aa4a-aba5013844dc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:16.169397 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db66a17-d011-4814-aa4a-aba5013844dc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:16.175241 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aff85c-cc05-4cac-99aa-7bd880475fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:16.175917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aff85c-cc05-4cac-99aa-7bd880475fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:16.176359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aff85c-cc05-4cac-99aa-7bd880475fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:16.176648 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aff85c-cc05-4cac-99aa-7bd880475fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:16.177384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aff85c-cc05-4cac-99aa-7bd880475fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:16.179142 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aff85c-cc05-4cac-99aa-7bd880475fe7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:16.191002 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 931/1845] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:16.195046 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6030bf4-b93a-452b-8522-6d52a155c50e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:16.205531 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6030bf4-b93a-452b-8522-6d52a155c50e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:16.210152 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2f238-a3a1-49d3-ad45-d476a72985f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:16.211018 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2f238-a3a1-49d3-ad45-d476a72985f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:16.211297 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2f238-a3a1-49d3-ad45-d476a72985f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:16.211568 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2f238-a3a1-49d3-ad45-d476a72985f8 admin admin] PATH_INFO: `/v3/projects/c142afa3e0634f2a82908733311b799e/users/fb6716df110747669d0f392b2026cefb/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:16.229102 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2f238-a3a1-49d3-ad45-d476a72985f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c142afa3e0634f2a82908733311b799e, user_id=fb6716df110747669d0f392b2026cefb, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:16.231800 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2f238-a3a1-49d3-ad45-d476a72985f8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:16.246458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 915/1846] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:16 2022] PUT /identity/v3/projects/c142afa3e0634f2a82908733311b799e/users/fb6716df110747669d0f392b2026cefb/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:16.256783 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001415d-c57f-484c-a81c-8adc7d83e7cc None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:16.257083 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001415d-c57f-484c-a81c-8adc7d83e7cc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:16.257401 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001415d-c57f-484c-a81c-8adc7d83e7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:16.300797 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0001415d-c57f-484c-a81c-8adc7d83e7cc None None] MFA Rules not processed for user `fb6716df110747669d0f392b2026cefb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:16.302959 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0001415d-c57f-484c-a81c-8adc7d83e7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:16.395798 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 932/1847] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:16.400316 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2f238-a3a1-49d3-ad45-d476a72985f8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:16.444557 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:16.564663 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2f238-a3a1-49d3-ad45-d476a72985f8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:16.572756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988af82-4685-49fb-8b0f-5fc5415ddffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:16.573415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988af82-4685-49fb-8b0f-5fc5415ddffa admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:16.573792 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988af82-4685-49fb-8b0f-5fc5415ddffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:16.574136 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988af82-4685-49fb-8b0f-5fc5415ddffa admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:16.575178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988af82-4685-49fb-8b0f-5fc5415ddffa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:16.578151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988af82-4685-49fb-8b0f-5fc5415ddffa admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:16.618927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0001415d-c57f-484c-a81c-8adc7d83e7cc None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:16.626540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 916/1848] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:16 2022] POST /identity/v3/projects => generated 379 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:16.634009 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0001415d-c57f-484c-a81c-8adc7d83e7cc None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:16.688449 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:16.722398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a3f66-fd82-4319-9e00-f34d7f7702c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:16.723204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a3f66-fd82-4319-9e00-f34d7f7702c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:16.723499 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a3f66-fd82-4319-9e00-f34d7f7702c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:16.723782 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a3f66-fd82-4319-9e00-f34d7f7702c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:16.724476 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a3f66-fd82-4319-9e00-f34d7f7702c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:16.726447 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a3f66-fd82-4319-9e00-f34d7f7702c3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:16.751250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 933/1849] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:16 2022] POST /identity/v3/users => generated 296 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:16.755440 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988af82-4685-49fb-8b0f-5fc5415ddffa admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:16.768158 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988af82-4685-49fb-8b0f-5fc5415ddffa admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:16.780011 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e69625-cfa4-4f3d-addb-3947c6b95dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:16.781007 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e69625-cfa4-4f3d-addb-3947c6b95dac admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:16.781429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e69625-cfa4-4f3d-addb-3947c6b95dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:16.781743 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e69625-cfa4-4f3d-addb-3947c6b95dac admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:16.782980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e69625-cfa4-4f3d-addb-3947c6b95dac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:16.784517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e69625-cfa4-4f3d-addb-3947c6b95dac admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:16.801813 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 917/1850] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:16.803481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a3f66-fd82-4319-9e00-f34d7f7702c3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:16.818162 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a3f66-fd82-4319-9e00-f34d7f7702c3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:16.825194 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08270adc-210e-4fdf-91e8-67f319e2e1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:16.828415 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08270adc-210e-4fdf-91e8-67f319e2e1bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:16.828415 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08270adc-210e-4fdf-91e8-67f319e2e1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:16.828415 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08270adc-210e-4fdf-91e8-67f319e2e1bc admin admin] PATH_INFO: `/v3/projects/455438b02ad34d45af2d06bd0bf6a86d/users/7189c464b2a04cb286efcfcc4f389ef5/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:16.856416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08270adc-210e-4fdf-91e8-67f319e2e1bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=455438b02ad34d45af2d06bd0bf6a86d, user_id=7189c464b2a04cb286efcfcc4f389ef5, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:16.860287 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08270adc-210e-4fdf-91e8-67f319e2e1bc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:16.898602 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 934/1851] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:16 2022] PUT /identity/v3/projects/455438b02ad34d45af2d06bd0bf6a86d/users/7189c464b2a04cb286efcfcc4f389ef5/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:16.902433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e69625-cfa4-4f3d-addb-3947c6b95dac admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:16.918954 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e69625-cfa4-4f3d-addb-3947c6b95dac admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:17.005435 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6cec0-8c04-4ffa-8938-4effafe03734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:17.006145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6cec0-8c04-4ffa-8938-4effafe03734 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:17.006433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6cec0-8c04-4ffa-8938-4effafe03734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:17.006814 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6cec0-8c04-4ffa-8938-4effafe03734 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:17.007435 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6cec0-8c04-4ffa-8938-4effafe03734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:17.009030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6cec0-8c04-4ffa-8938-4effafe03734 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:17.023327 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 918/1852] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:17.027699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08270adc-210e-4fdf-91e8-67f319e2e1bc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:17.042497 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08270adc-210e-4fdf-91e8-67f319e2e1bc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:17.047354 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e0540-963c-4d08-97b1-3b59d5466bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:17.048242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e0540-963c-4d08-97b1-3b59d5466bba admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:17.048529 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e0540-963c-4d08-97b1-3b59d5466bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:17.048826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e0540-963c-4d08-97b1-3b59d5466bba admin admin] PATH_INFO: `/v3/projects/455438b02ad34d45af2d06bd0bf6a86d/users/7189c464b2a04cb286efcfcc4f389ef5/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:17.050814 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e0540-963c-4d08-97b1-3b59d5466bba admin admin] RBAC: Authorizing `identity:create_grant(project_id=455438b02ad34d45af2d06bd0bf6a86d, user_id=7189c464b2a04cb286efcfcc4f389ef5, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:17.053467 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e0540-963c-4d08-97b1-3b59d5466bba admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:17.065191 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 935/1853] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:17 2022] PUT /identity/v3/projects/455438b02ad34d45af2d06bd0bf6a86d/users/7189c464b2a04cb286efcfcc4f389ef5/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:17.076883 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed60e49-93a0-4311-a188-dc5812bdec0d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:17.077206 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed60e49-93a0-4311-a188-dc5812bdec0d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:17.077638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed60e49-93a0-4311-a188-dc5812bdec0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:17.103123 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-aed60e49-93a0-4311-a188-dc5812bdec0d None None] MFA Rules not processed for user `7189c464b2a04cb286efcfcc4f389ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:17.104338 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-aed60e49-93a0-4311-a188-dc5812bdec0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:17.170751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 919/1854] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:17 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:17.182520 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e0540-963c-4d08-97b1-3b59d5466bba admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:17.184251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c99e0540-963c-4d08-97b1-3b59d5466bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:17.256852 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e0540-963c-4d08-97b1-3b59d5466bba admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:17.261042 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a206489-b072-4086-a8df-144725da7929 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:17.261840 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a206489-b072-4086-a8df-144725da7929 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:17.262226 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a206489-b072-4086-a8df-144725da7929 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:17.262756 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a206489-b072-4086-a8df-144725da7929 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:17.274629 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a206489-b072-4086-a8df-144725da7929 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:17.276176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a206489-b072-4086-a8df-144725da7929 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:17.286769 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 936/1855] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:17 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:17.615640 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed60e49-93a0-4311-a188-dc5812bdec0d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:17.625733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed60e49-93a0-4311-a188-dc5812bdec0d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:17.630331 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d6c79-b34b-4c0e-b526-67671599ffaf tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7189c464b2a04cb286efcfcc4f389ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '455438b02ad34d45af2d06bd0bf6a86d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:17.630978 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d6c79-b34b-4c0e-b526-67671599ffaf tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:17.631264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d6c79-b34b-4c0e-b526-67671599ffaf tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:17.631541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d6c79-b34b-4c0e-b526-67671599ffaf tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:17.632263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d6c79-b34b-4c0e-b526-67671599ffaf tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:29:17.632572 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d6c79-b34b-4c0e-b526-67671599ffaf tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:17.634196 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d6c79-b34b-4c0e-b526-67671599ffaf tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:17.641623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:17.648365 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 920/1856] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:29:17.652188 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a206489-b072-4086-a8df-144725da7929 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:17.661380 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a206489-b072-4086-a8df-144725da7929 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:17.665926 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6650df-97c7-4c44-89c3-9bbc25dec956 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7189c464b2a04cb286efcfcc4f389ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '455438b02ad34d45af2d06bd0bf6a86d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:17.666562 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6650df-97c7-4c44-89c3-9bbc25dec956 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:17.666850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6650df-97c7-4c44-89c3-9bbc25dec956 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:17.667133 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6650df-97c7-4c44-89c3-9bbc25dec956 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:17.667823 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6650df-97c7-4c44-89c3-9bbc25dec956 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:17.669776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6650df-97c7-4c44-89c3-9bbc25dec956 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:17.697354 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 937/1857] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:17 2022] POST /identity/v3/projects => generated 353 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:29:17.702447 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d6c79-b34b-4c0e-b526-67671599ffaf tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:17.713222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d6c79-b34b-4c0e-b526-67671599ffaf tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:17.771397 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f319e53-2aeb-4474-a11a-87f30403ab47 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7189c464b2a04cb286efcfcc4f389ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '455438b02ad34d45af2d06bd0bf6a86d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:17.772085 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f319e53-2aeb-4474-a11a-87f30403ab47 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:17.772364 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f319e53-2aeb-4474-a11a-87f30403ab47 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:17.772643 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f319e53-2aeb-4474-a11a-87f30403ab47 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:17.773367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f319e53-2aeb-4474-a11a-87f30403ab47 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:29:17.773707 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f319e53-2aeb-4474-a11a-87f30403ab47 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:17.775321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f319e53-2aeb-4474-a11a-87f30403ab47 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:17.787071 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 921/1858] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:29:17.825961 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6650df-97c7-4c44-89c3-9bbc25dec956 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:17.837254 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6650df-97c7-4c44-89c3-9bbc25dec956 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:17.842827 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfecb8ac-102b-433a-bb4d-fe72041ffe48 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7189c464b2a04cb286efcfcc4f389ef5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '455438b02ad34d45af2d06bd0bf6a86d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:17.843670 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecb8ac-102b-433a-bb4d-fe72041ffe48 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:17.843957 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecb8ac-102b-433a-bb4d-fe72041ffe48 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:17.844241 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecb8ac-102b-433a-bb4d-fe72041ffe48 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] PATH_INFO: `/v3/projects/692223d055664da394dbb5cd33496dbb` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:17.845437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfecb8ac-102b-433a-bb4d-fe72041ffe48 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] RBAC: Authorizing `identity:delete_project(project_id=692223d055664da394dbb5cd33496dbb)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:17.847553 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfecb8ac-102b-433a-bb4d-fe72041ffe48 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:17.893453 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bfecb8ac-102b-433a-bb4d-fe72041ffe48 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '692223d055664da394dbb5cd33496dbb', 'request_id': 'req-bfecb8ac-102b-433a-bb4d-fe72041ffe48'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:17.899922 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bfecb8ac-102b-433a-bb4d-fe72041ffe48 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '692223d055664da394dbb5cd33496dbb', 'request_id': 'req-bfecb8ac-102b-433a-bb4d-fe72041ffe48'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:17.903619 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:INSERT=6,keystone:DELETE=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:17.907384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bfecb8ac-102b-433a-bb4d-fe72041ffe48 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '692223d055664da394dbb5cd33496dbb', 'request_id': 'req-bfecb8ac-102b-433a-bb4d-fe72041ffe48'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:17.921848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bfecb8ac-102b-433a-bb4d-fe72041ffe48 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] The token cache is being invalidate because project 692223d055664da394dbb5cd33496dbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:17.922222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bfecb8ac-102b-433a-bb4d-fe72041ffe48 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:17.922911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bfecb8ac-102b-433a-bb4d-fe72041ffe48 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:17.924777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 938/1859] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:17 2022] DELETE /identity/v3/projects/692223d055664da394dbb5cd33496dbb => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:18.141439 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f319e53-2aeb-4474-a11a-87f30403ab47 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:18.143258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3f319e53-2aeb-4474-a11a-87f30403ab47 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:18.207599 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f319e53-2aeb-4474-a11a-87f30403ab47 tempest-VolumeQuotaClassesTest-692718182 tempest-VolumeQuotaClassesTest-692718182-project-admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:18.262047 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a05a5b-add6-4b63-a71f-82ef885fad8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:18.262902 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a05a5b-add6-4b63-a71f-82ef885fad8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:18.263184 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a05a5b-add6-4b63-a71f-82ef885fad8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:18.263463 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a05a5b-add6-4b63-a71f-82ef885fad8b admin admin] PATH_INFO: `/v3/users/fb6716df110747669d0f392b2026cefb` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:18.264790 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a05a5b-add6-4b63-a71f-82ef885fad8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb6716df110747669d0f392b2026cefb)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:18.266727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a05a5b-add6-4b63-a71f-82ef885fad8b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:18.307355 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:18.313070 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-06a05a5b-add6-4b63-a71f-82ef885fad8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb6716df110747669d0f392b2026cefb'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:18.314062 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-06a05a5b-add6-4b63-a71f-82ef885fad8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb6716df110747669d0f392b2026cefb'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:18.326003 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-06a05a5b-add6-4b63-a71f-82ef885fad8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb6716df110747669d0f392b2026cefb'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:18.333096 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-06a05a5b-add6-4b63-a71f-82ef885fad8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb6716df110747669d0f392b2026cefb'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:18.333746 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-06a05a5b-add6-4b63-a71f-82ef885fad8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb6716df110747669d0f392b2026cefb'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:18.346676 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 922/1860] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:18 2022] DELETE /identity/v3/users/fb6716df110747669d0f392b2026cefb => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:18.351090 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec938dd1-0880-4423-8941-59a834a65b12 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:18.351384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec938dd1-0880-4423-8941-59a834a65b12 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:18.351663 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec938dd1-0880-4423-8941-59a834a65b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:18.368974 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ec938dd1-0880-4423-8941-59a834a65b12 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:18.370197 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ec938dd1-0880-4423-8941-59a834a65b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:18.428196 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 939/1861] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:18.438131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a05a5b-add6-4b63-a71f-82ef885fad8b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:18.439529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-06a05a5b-add6-4b63-a71f-82ef885fad8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:18.500930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a05a5b-add6-4b63-a71f-82ef885fad8b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:18.505515 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a46547-d0f6-4305-b01a-8fb694f420d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:18.506187 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a46547-d0f6-4305-b01a-8fb694f420d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:18.506468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a46547-d0f6-4305-b01a-8fb694f420d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:18.506872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a46547-d0f6-4305-b01a-8fb694f420d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:18.516653 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a46547-d0f6-4305-b01a-8fb694f420d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:18.518159 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a46547-d0f6-4305-b01a-8fb694f420d8 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:18.527634 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 923/1862] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:29:18.958126 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec938dd1-0880-4423-8941-59a834a65b12 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:18.960162 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ec938dd1-0880-4423-8941-59a834a65b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:18.991189 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec938dd1-0880-4423-8941-59a834a65b12 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:19.014738 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e0959-4eb9-406e-a180-73894201df4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:19.015632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e0959-4eb9-406e-a180-73894201df4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:19.015923 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e0959-4eb9-406e-a180-73894201df4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:19.016207 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e0959-4eb9-406e-a180-73894201df4d admin admin] PATH_INFO: `/v3/projects/c142afa3e0634f2a82908733311b799e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:19.021226 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e0959-4eb9-406e-a180-73894201df4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c142afa3e0634f2a82908733311b799e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:19.023232 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e0959-4eb9-406e-a180-73894201df4d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:19.137512 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-da7e0959-4eb9-406e-a180-73894201df4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c142afa3e0634f2a82908733311b799e', 'request_id': 'req-da7e0959-4eb9-406e-a180-73894201df4d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:19.149636 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-da7e0959-4eb9-406e-a180-73894201df4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c142afa3e0634f2a82908733311b799e', 'request_id': 'req-da7e0959-4eb9-406e-a180-73894201df4d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:19.158025 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-da7e0959-4eb9-406e-a180-73894201df4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c142afa3e0634f2a82908733311b799e', 'request_id': 'req-da7e0959-4eb9-406e-a180-73894201df4d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:19.224423 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-da7e0959-4eb9-406e-a180-73894201df4d admin admin] The token cache is being invalidate because project c142afa3e0634f2a82908733311b799e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:19.225740 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-da7e0959-4eb9-406e-a180-73894201df4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:19.226457 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-da7e0959-4eb9-406e-a180-73894201df4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:19.228138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 940/1863] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:18 2022] DELETE /identity/v3/projects/c142afa3e0634f2a82908733311b799e => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:19.233632 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a46547-d0f6-4305-b01a-8fb694f420d8 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:19.243575 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a46547-d0f6-4305-b01a-8fb694f420d8 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:19.307536 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c1d21-0a4b-40a9-ba34-26eddf9b64f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:19.308717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c1d21-0a4b-40a9-ba34-26eddf9b64f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:19.309188 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c1d21-0a4b-40a9-ba34-26eddf9b64f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:19.309533 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c1d21-0a4b-40a9-ba34-26eddf9b64f4 admin admin] PATH_INFO: `/v3/users/7189c464b2a04cb286efcfcc4f389ef5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:19.314434 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c1d21-0a4b-40a9-ba34-26eddf9b64f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7189c464b2a04cb286efcfcc4f389ef5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:19.314434 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c1d21-0a4b-40a9-ba34-26eddf9b64f4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:19.369141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fa2c1d21-0a4b-40a9-ba34-26eddf9b64f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7189c464b2a04cb286efcfcc4f389ef5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:19.369816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fa2c1d21-0a4b-40a9-ba34-26eddf9b64f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7189c464b2a04cb286efcfcc4f389ef5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:19.380446 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:19.381835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fa2c1d21-0a4b-40a9-ba34-26eddf9b64f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7189c464b2a04cb286efcfcc4f389ef5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:19.391988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fa2c1d21-0a4b-40a9-ba34-26eddf9b64f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7189c464b2a04cb286efcfcc4f389ef5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:19.393231 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fa2c1d21-0a4b-40a9-ba34-26eddf9b64f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7189c464b2a04cb286efcfcc4f389ef5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:19.406452 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 924/1864] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:19 2022] DELETE /identity/v3/users/7189c464b2a04cb286efcfcc4f389ef5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:19.684515 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e0959-4eb9-406e-a180-73894201df4d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:19.686630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-da7e0959-4eb9-406e-a180-73894201df4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:19.709121 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e0959-4eb9-406e-a180-73894201df4d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:19.777354 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12843ab3-7586-4341-944c-396062434ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:19.778291 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12843ab3-7586-4341-944c-396062434ef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:19.778575 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12843ab3-7586-4341-944c-396062434ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:19.778854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12843ab3-7586-4341-944c-396062434ef0 admin admin] PATH_INFO: `/v3/projects/455438b02ad34d45af2d06bd0bf6a86d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:19.780158 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12843ab3-7586-4341-944c-396062434ef0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=455438b02ad34d45af2d06bd0bf6a86d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:19.782568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12843ab3-7586-4341-944c-396062434ef0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:19.794298 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:19.833010 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-12843ab3-7586-4341-944c-396062434ef0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '455438b02ad34d45af2d06bd0bf6a86d', 'request_id': 'req-12843ab3-7586-4341-944c-396062434ef0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:19.839762 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-12843ab3-7586-4341-944c-396062434ef0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '455438b02ad34d45af2d06bd0bf6a86d', 'request_id': 'req-12843ab3-7586-4341-944c-396062434ef0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:19.847382 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-12843ab3-7586-4341-944c-396062434ef0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '455438b02ad34d45af2d06bd0bf6a86d', 'request_id': 'req-12843ab3-7586-4341-944c-396062434ef0'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:19.865889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-12843ab3-7586-4341-944c-396062434ef0 admin admin] The token cache is being invalidate because project 455438b02ad34d45af2d06bd0bf6a86d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:19.865889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-12843ab3-7586-4341-944c-396062434ef0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:19.866804 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-12843ab3-7586-4341-944c-396062434ef0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:19.868397 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 941/1865] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:19 2022] DELETE /identity/v3/projects/455438b02ad34d45af2d06bd0bf6a86d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:19.873912 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010883c-eb2c-4d61-ad2f-4cd79af37d77 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:19.874198 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010883c-eb2c-4d61-ad2f-4cd79af37d77 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:19.874476 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010883c-eb2c-4d61-ad2f-4cd79af37d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:19.892536 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c010883c-eb2c-4d61-ad2f-4cd79af37d77 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:19.893855 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c010883c-eb2c-4d61-ad2f-4cd79af37d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:19.956225 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 925/1866] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:19.964055 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30847f43-352a-4edb-b4fb-5566ad2990be None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:19.964560 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30847f43-352a-4edb-b4fb-5566ad2990be None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:19.964644 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30847f43-352a-4edb-b4fb-5566ad2990be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:19.984808 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-30847f43-352a-4edb-b4fb-5566ad2990be None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:19.995141 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-30847f43-352a-4edb-b4fb-5566ad2990be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:20.009069 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 942/1867] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:20.013282 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010883c-eb2c-4d61-ad2f-4cd79af37d77 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:20.023554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010883c-eb2c-4d61-ad2f-4cd79af37d77 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:20.028252 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5598d5c-03e4-4112-940a-bb90ae6104b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:20.028911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5598d5c-03e4-4112-940a-bb90ae6104b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:20.029729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5598d5c-03e4-4112-940a-bb90ae6104b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:20.030014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5598d5c-03e4-4112-940a-bb90ae6104b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:20.030734 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5598d5c-03e4-4112-940a-bb90ae6104b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:29:20.031043 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5598d5c-03e4-4112-940a-bb90ae6104b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:20.032668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5598d5c-03e4-4112-940a-bb90ae6104b5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:20.044302 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 926/1868] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:29:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:29:20.049224 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30847f43-352a-4edb-b4fb-5566ad2990be None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:20.058363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30847f43-352a-4edb-b4fb-5566ad2990be None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:20.063986 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252c9b0-484a-4b55-adf7-d11c207b24c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:20.064638 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252c9b0-484a-4b55-adf7-d11c207b24c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:20.064928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252c9b0-484a-4b55-adf7-d11c207b24c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:20.065208 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252c9b0-484a-4b55-adf7-d11c207b24c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:20.065946 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252c9b0-484a-4b55-adf7-d11c207b24c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:20.069873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252c9b0-484a-4b55-adf7-d11c207b24c2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:20.101302 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 943/1869] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:20 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:20.102428 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5598d5c-03e4-4112-940a-bb90ae6104b5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:20.113801 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5598d5c-03e4-4112-940a-bb90ae6104b5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:20.173331 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc908873-6435-419c-b218-666f26d93162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:20.174034 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc908873-6435-419c-b218-666f26d93162 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:20.174325 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc908873-6435-419c-b218-666f26d93162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:20.174606 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc908873-6435-419c-b218-666f26d93162 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:20.175297 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc908873-6435-419c-b218-666f26d93162 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:20.177208 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc908873-6435-419c-b218-666f26d93162 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:20.199978 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 927/1870] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:20 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:20.201983 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252c9b0-484a-4b55-adf7-d11c207b24c2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:20.212522 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252c9b0-484a-4b55-adf7-d11c207b24c2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:20.217966 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c434cd-0eab-4b1a-aa04-3ca1e9d902ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:20.218667 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c434cd-0eab-4b1a-aa04-3ca1e9d902ee admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:20.219089 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c434cd-0eab-4b1a-aa04-3ca1e9d902ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:20.219383 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c434cd-0eab-4b1a-aa04-3ca1e9d902ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:20.220117 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c434cd-0eab-4b1a-aa04-3ca1e9d902ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:20.221753 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c434cd-0eab-4b1a-aa04-3ca1e9d902ee admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:20.233256 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 944/1871] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:20.236906 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc908873-6435-419c-b218-666f26d93162 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:20.250112 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc908873-6435-419c-b218-666f26d93162 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:20.255545 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11eb54-f8de-4906-aa79-b182b96a184d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:20.256220 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11eb54-f8de-4906-aa79-b182b96a184d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:20.256513 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11eb54-f8de-4906-aa79-b182b96a184d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:20.256801 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11eb54-f8de-4906-aa79-b182b96a184d admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:20.257610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11eb54-f8de-4906-aa79-b182b96a184d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:20.259529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11eb54-f8de-4906-aa79-b182b96a184d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:20.272816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 928/1872] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:20.277773 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c434cd-0eab-4b1a-aa04-3ca1e9d902ee admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:20.288401 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c434cd-0eab-4b1a-aa04-3ca1e9d902ee admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:20.293234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690afd5-8e3d-452e-8ad9-4bb7135503ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:20.294144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690afd5-8e3d-452e-8ad9-4bb7135503ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:20.294435 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690afd5-8e3d-452e-8ad9-4bb7135503ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:20.294716 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690afd5-8e3d-452e-8ad9-4bb7135503ef admin admin] PATH_INFO: `/v3/projects/3d68fb8e05e24571bed8fddcabd150b3/users/bc52add47eaf4d2bbad3e7f05b53cc83/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:20.310402 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2690afd5-8e3d-452e-8ad9-4bb7135503ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d68fb8e05e24571bed8fddcabd150b3, user_id=bc52add47eaf4d2bbad3e7f05b53cc83, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:20.313944 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2690afd5-8e3d-452e-8ad9-4bb7135503ef admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:20.328601 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 945/1873] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:20 2022] PUT /identity/v3/projects/3d68fb8e05e24571bed8fddcabd150b3/users/bc52add47eaf4d2bbad3e7f05b53cc83/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:20.338863 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb107071-a416-4f23-b6b4-84ae0e769ac8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:20.339154 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb107071-a416-4f23-b6b4-84ae0e769ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:20.339437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb107071-a416-4f23-b6b4-84ae0e769ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:20.366840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-eb107071-a416-4f23-b6b4-84ae0e769ac8 None None] MFA Rules not processed for user `bc52add47eaf4d2bbad3e7f05b53cc83`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:20.368186 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-eb107071-a416-4f23-b6b4-84ae0e769ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:20.402244 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:20.434925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 929/1874] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:20.439652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690afd5-8e3d-452e-8ad9-4bb7135503ef admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:20.503183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690afd5-8e3d-452e-8ad9-4bb7135503ef admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:20.507868 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf6b3a-cf94-4493-8738-606a3b5b613b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:20.508520 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf6b3a-cf94-4493-8738-606a3b5b613b admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:20.508804 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf6b3a-cf94-4493-8738-606a3b5b613b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:20.509078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf6b3a-cf94-4493-8738-606a3b5b613b admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:20.509787 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf6b3a-cf94-4493-8738-606a3b5b613b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:20.511693 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf6b3a-cf94-4493-8738-606a3b5b613b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:20.537041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 946/1875] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:20 2022] POST /identity/v3/projects => generated 389 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:20.541043 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb107071-a416-4f23-b6b4-84ae0e769ac8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:20.552806 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb107071-a416-4f23-b6b4-84ae0e769ac8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:20.608525 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2927d1-b865-42b9-ac20-aa51c7cf12fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:20.609191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2927d1-b865-42b9-ac20-aa51c7cf12fe admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:20.609479 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2927d1-b865-42b9-ac20-aa51c7cf12fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:20.609789 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2927d1-b865-42b9-ac20-aa51c7cf12fe admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:20.610477 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2927d1-b865-42b9-ac20-aa51c7cf12fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:20.612413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2927d1-b865-42b9-ac20-aa51c7cf12fe admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:20.631930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 930/1876] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:20 2022] POST /identity/v3/users => generated 301 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:20.636041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf6b3a-cf94-4493-8738-606a3b5b613b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:20.646459 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf6b3a-cf94-4493-8738-606a3b5b613b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:20.651071 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e5cd0-a1dd-46af-a386-ed290de6a7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:20.651743 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e5cd0-a1dd-46af-a386-ed290de6a7cc admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:20.652011 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e5cd0-a1dd-46af-a386-ed290de6a7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:20.652294 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e5cd0-a1dd-46af-a386-ed290de6a7cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:20.653010 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e5cd0-a1dd-46af-a386-ed290de6a7cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:20.655213 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e5cd0-a1dd-46af-a386-ed290de6a7cc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:20.665498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 947/1877] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:20.669424 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2927d1-b865-42b9-ac20-aa51c7cf12fe admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:20.680344 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2927d1-b865-42b9-ac20-aa51c7cf12fe admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:20.685177 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde21dca-df0b-4451-b2af-7a9f176891f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:20.686090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde21dca-df0b-4451-b2af-7a9f176891f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:20.686372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde21dca-df0b-4451-b2af-7a9f176891f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:20.686651 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde21dca-df0b-4451-b2af-7a9f176891f3 admin admin] PATH_INFO: `/v3/projects/7589e14efc694d63b0206b6022251c86/users/fd1be4040f1d4e538a1761ebd479da41/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:20.703431 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde21dca-df0b-4451-b2af-7a9f176891f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7589e14efc694d63b0206b6022251c86, user_id=fd1be4040f1d4e538a1761ebd479da41, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:20.706651 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde21dca-df0b-4451-b2af-7a9f176891f3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:20.722502 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 931/1878] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:20 2022] PUT /identity/v3/projects/7589e14efc694d63b0206b6022251c86/users/fd1be4040f1d4e538a1761ebd479da41/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:20.726605 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e5cd0-a1dd-46af-a386-ed290de6a7cc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:20.737598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e5cd0-a1dd-46af-a386-ed290de6a7cc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:20.745358 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:20.798643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f861d1-78f5-4239-a25c-755d3a3ca9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:20.799232 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f861d1-78f5-4239-a25c-755d3a3ca9b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:20.799524 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f861d1-78f5-4239-a25c-755d3a3ca9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:20.799805 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f861d1-78f5-4239-a25c-755d3a3ca9b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:20.800528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f861d1-78f5-4239-a25c-755d3a3ca9b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:20.802166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f861d1-78f5-4239-a25c-755d3a3ca9b0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:20.811747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 948/1879] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:20.816290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde21dca-df0b-4451-b2af-7a9f176891f3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:20.826629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde21dca-df0b-4451-b2af-7a9f176891f3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:20.831189 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e0fb3-1416-45ce-8be9-127b9cc24e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:20.832049 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e0fb3-1416-45ce-8be9-127b9cc24e98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:20.832330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e0fb3-1416-45ce-8be9-127b9cc24e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:20.832602 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e0fb3-1416-45ce-8be9-127b9cc24e98 admin admin] PATH_INFO: `/v3/projects/7589e14efc694d63b0206b6022251c86/users/fd1be4040f1d4e538a1761ebd479da41/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:20.834581 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e0fb3-1416-45ce-8be9-127b9cc24e98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7589e14efc694d63b0206b6022251c86, user_id=fd1be4040f1d4e538a1761ebd479da41, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:20.837205 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e0fb3-1416-45ce-8be9-127b9cc24e98 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:20.847860 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 932/1880] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:20 2022] PUT /identity/v3/projects/7589e14efc694d63b0206b6022251c86/users/fd1be4040f1d4e538a1761ebd479da41/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:20.859444 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03db47d-f79b-4891-98cf-0f0552a22598 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:20.859744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03db47d-f79b-4891-98cf-0f0552a22598 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:20.860031 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03db47d-f79b-4891-98cf-0f0552a22598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:20.885847 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b03db47d-f79b-4891-98cf-0f0552a22598 None None] MFA Rules not processed for user `fd1be4040f1d4e538a1761ebd479da41`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:20.887068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b03db47d-f79b-4891-98cf-0f0552a22598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:20.954504 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 949/1881] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:20.965383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e0fb3-1416-45ce-8be9-127b9cc24e98 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:20.967812 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a38e0fb3-1416-45ce-8be9-127b9cc24e98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:21.027752 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e0fb3-1416-45ce-8be9-127b9cc24e98 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:21.031996 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a1e0a-798e-4496-a779-a9f0f73241a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:21.032712 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a1e0a-798e-4496-a779-a9f0f73241a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:21.033066 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a1e0a-798e-4496-a779-a9f0f73241a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:21.033409 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a1e0a-798e-4496-a779-a9f0f73241a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:21.045470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a1e0a-798e-4496-a779-a9f0f73241a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:21.047153 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a1e0a-798e-4496-a779-a9f0f73241a7 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:21.057155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 933/1882] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:20 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:21.203853 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03db47d-f79b-4891-98cf-0f0552a22598 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:21.280809 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03db47d-f79b-4891-98cf-0f0552a22598 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:21.286322 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e46f71-b3ce-4e74-b061-b2bc2465f97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:21.287225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e46f71-b3ce-4e74-b061-b2bc2465f97e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:21.287631 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e46f71-b3ce-4e74-b061-b2bc2465f97e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:21.287780 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e46f71-b3ce-4e74-b061-b2bc2465f97e admin admin] PATH_INFO: `/v3/users/bc52add47eaf4d2bbad3e7f05b53cc83` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:21.289042 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e46f71-b3ce-4e74-b061-b2bc2465f97e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc52add47eaf4d2bbad3e7f05b53cc83)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:21.291333 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e46f71-b3ce-4e74-b061-b2bc2465f97e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:21.344031 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:21.365670 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-61e46f71-b3ce-4e74-b061-b2bc2465f97e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc52add47eaf4d2bbad3e7f05b53cc83'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:21.379045 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-61e46f71-b3ce-4e74-b061-b2bc2465f97e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc52add47eaf4d2bbad3e7f05b53cc83'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:21.379852 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-61e46f71-b3ce-4e74-b061-b2bc2465f97e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc52add47eaf4d2bbad3e7f05b53cc83'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:21.387620 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-61e46f71-b3ce-4e74-b061-b2bc2465f97e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc52add47eaf4d2bbad3e7f05b53cc83'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:21.388511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-61e46f71-b3ce-4e74-b061-b2bc2465f97e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc52add47eaf4d2bbad3e7f05b53cc83'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:21.403452 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 950/1883] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:21 2022] DELETE /identity/v3/users/bc52add47eaf4d2bbad3e7f05b53cc83 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:21.408515 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc977d60-f8fe-4303-a0fe-52bd34dfe938 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:21.409012 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc977d60-f8fe-4303-a0fe-52bd34dfe938 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:21.409099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc977d60-f8fe-4303-a0fe-52bd34dfe938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:21.441286 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-bc977d60-f8fe-4303-a0fe-52bd34dfe938 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:21.442269 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bc977d60-f8fe-4303-a0fe-52bd34dfe938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:21.470155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:21.505375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 934/1884] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:21.517423 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e46f71-b3ce-4e74-b061-b2bc2465f97e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:21.519067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-61e46f71-b3ce-4e74-b061-b2bc2465f97e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:21.580203 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e46f71-b3ce-4e74-b061-b2bc2465f97e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:21.583990 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7734431-39f6-4116-8447-c0035c24c4e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:21.584624 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7734431-39f6-4116-8447-c0035c24c4e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:21.584902 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7734431-39f6-4116-8447-c0035c24c4e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:21.585179 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7734431-39f6-4116-8447-c0035c24c4e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:21.595943 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7734431-39f6-4116-8447-c0035c24c4e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:21.597392 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7734431-39f6-4116-8447-c0035c24c4e5 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:21.609243 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 951/1885] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:21.885961 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc977d60-f8fe-4303-a0fe-52bd34dfe938 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:21.887622 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bc977d60-f8fe-4303-a0fe-52bd34dfe938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:21.901278 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc977d60-f8fe-4303-a0fe-52bd34dfe938 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:21.911569 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05865d2c-0885-4003-83ee-182e8bbe09c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:21.912457 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05865d2c-0885-4003-83ee-182e8bbe09c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:21.912744 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05865d2c-0885-4003-83ee-182e8bbe09c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:21.913031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05865d2c-0885-4003-83ee-182e8bbe09c5 admin admin] PATH_INFO: `/v3/projects/3d68fb8e05e24571bed8fddcabd150b3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:21.914392 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05865d2c-0885-4003-83ee-182e8bbe09c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d68fb8e05e24571bed8fddcabd150b3)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:21.916344 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05865d2c-0885-4003-83ee-182e8bbe09c5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:21.969934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-05865d2c-0885-4003-83ee-182e8bbe09c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d68fb8e05e24571bed8fddcabd150b3', 'request_id': 'req-05865d2c-0885-4003-83ee-182e8bbe09c5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:21.976451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-05865d2c-0885-4003-83ee-182e8bbe09c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d68fb8e05e24571bed8fddcabd150b3', 'request_id': 'req-05865d2c-0885-4003-83ee-182e8bbe09c5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:21.981967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-05865d2c-0885-4003-83ee-182e8bbe09c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d68fb8e05e24571bed8fddcabd150b3', 'request_id': 'req-05865d2c-0885-4003-83ee-182e8bbe09c5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:21.994692 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-05865d2c-0885-4003-83ee-182e8bbe09c5 admin admin] The token cache is being invalidate because project 3d68fb8e05e24571bed8fddcabd150b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:21.994984 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-05865d2c-0885-4003-83ee-182e8bbe09c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:21.995620 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-05865d2c-0885-4003-83ee-182e8bbe09c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:21.997164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 935/1886] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:21 2022] DELETE /identity/v3/projects/3d68fb8e05e24571bed8fddcabd150b3 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:22.001224 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7734431-39f6-4116-8447-c0035c24c4e5 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:22.011119 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7734431-39f6-4116-8447-c0035c24c4e5 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:22.073653 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56f09a-a032-48ed-b41d-780dd4b09ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:22.074530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56f09a-a032-48ed-b41d-780dd4b09ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:22.074814 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56f09a-a032-48ed-b41d-780dd4b09ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:22.075129 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56f09a-a032-48ed-b41d-780dd4b09ab8 admin admin] PATH_INFO: `/v3/users/fd1be4040f1d4e538a1761ebd479da41` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:22.076298 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56f09a-a032-48ed-b41d-780dd4b09ab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd1be4040f1d4e538a1761ebd479da41)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:22.079800 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56f09a-a032-48ed-b41d-780dd4b09ab8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:22.128408 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bd56f09a-a032-48ed-b41d-780dd4b09ab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd1be4040f1d4e538a1761ebd479da41'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:22.136545 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:22.146306 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bd56f09a-a032-48ed-b41d-780dd4b09ab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd1be4040f1d4e538a1761ebd479da41'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:22.147320 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bd56f09a-a032-48ed-b41d-780dd4b09ab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd1be4040f1d4e538a1761ebd479da41'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:22.155156 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bd56f09a-a032-48ed-b41d-780dd4b09ab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd1be4040f1d4e538a1761ebd479da41'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:22.155843 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bd56f09a-a032-48ed-b41d-780dd4b09ab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd1be4040f1d4e538a1761ebd479da41'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:22.169296 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 952/1887] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:22 2022] DELETE /identity/v3/users/fd1be4040f1d4e538a1761ebd479da41 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:22.367489 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e53fb-be41-4831-ae3d-48cf7d890df5 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:22.367776 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e53fb-be41-4831-ae3d-48cf7d890df5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:22.368119 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e53fb-be41-4831-ae3d-48cf7d890df5 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:22.369501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 936/1888] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 30 18:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:29:22.386077 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536683a9-b41a-48b0-8d1a-87853d7843f1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:22.386593 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536683a9-b41a-48b0-8d1a-87853d7843f1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:22.386593 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536683a9-b41a-48b0-8d1a-87853d7843f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:22.412190 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-536683a9-b41a-48b0-8d1a-87853d7843f1 None None] MFA Rules not processed for user `b8751a4e92174d18b7b138392d2579a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:22.413488 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-536683a9-b41a-48b0-8d1a-87853d7843f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:22.533248 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e53fb-be41-4831-ae3d-48cf7d890df5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:22.536843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-028e53fb-be41-4831-ae3d-48cf7d890df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:22.609414 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 953/1889] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 30 18:29:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:22.666180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e53fb-be41-4831-ae3d-48cf7d890df5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:22.673387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c6a490-b7ed-431f-b6bf-6b71b4dd7f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:22.675289 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c6a490-b7ed-431f-b6bf-6b71b4dd7f50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:22.675792 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c6a490-b7ed-431f-b6bf-6b71b4dd7f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:22.676270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c6a490-b7ed-431f-b6bf-6b71b4dd7f50 admin admin] PATH_INFO: `/v3/projects/7589e14efc694d63b0206b6022251c86` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:22.677567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c6a490-b7ed-431f-b6bf-6b71b4dd7f50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7589e14efc694d63b0206b6022251c86)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:22.680108 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c6a490-b7ed-431f-b6bf-6b71b4dd7f50 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:22.778506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-23c6a490-b7ed-431f-b6bf-6b71b4dd7f50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7589e14efc694d63b0206b6022251c86', 'request_id': 'req-23c6a490-b7ed-431f-b6bf-6b71b4dd7f50'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:22.785952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-23c6a490-b7ed-431f-b6bf-6b71b4dd7f50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7589e14efc694d63b0206b6022251c86', 'request_id': 'req-23c6a490-b7ed-431f-b6bf-6b71b4dd7f50'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:22.788508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:22.791341 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536683a9-b41a-48b0-8d1a-87853d7843f1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:22.793051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-536683a9-b41a-48b0-8d1a-87853d7843f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:22.799358 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-23c6a490-b7ed-431f-b6bf-6b71b4dd7f50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7589e14efc694d63b0206b6022251c86', 'request_id': 'req-23c6a490-b7ed-431f-b6bf-6b71b4dd7f50'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:22.810331 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536683a9-b41a-48b0-8d1a-87853d7843f1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:22.819847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-23c6a490-b7ed-431f-b6bf-6b71b4dd7f50 admin admin] The token cache is being invalidate because project 7589e14efc694d63b0206b6022251c86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:22.820315 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-23c6a490-b7ed-431f-b6bf-6b71b4dd7f50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:22.821301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-23c6a490-b7ed-431f-b6bf-6b71b4dd7f50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:22.823016 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 937/1890] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:22 2022] DELETE /identity/v3/projects/7589e14efc694d63b0206b6022251c86 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:22.828585 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2fec4-b4b2-4965-b866-d6f384f1395f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:22.829002 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2fec4-b4b2-4965-b866-d6f384f1395f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:22.829290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2fec4-b4b2-4965-b866-d6f384f1395f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:22.867439 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-01c2fec4-b4b2-4965-b866-d6f384f1395f None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:22.869383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-01c2fec4-b4b2-4965-b866-d6f384f1395f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:22.894895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6c7f5-4c08-4ccc-924c-5f8276f4eec2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:22.895614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6c7f5-4c08-4ccc-924c-5f8276f4eec2 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:22.896581 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6c7f5-4c08-4ccc-924c-5f8276f4eec2 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:22.896581 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6c7f5-4c08-4ccc-924c-5f8276f4eec2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:22.899754 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7cf6c7f5-4c08-4ccc-924c-5f8276f4eec2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:22.916271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6c7f5-4c08-4ccc-924c-5f8276f4eec2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:22.918008 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6c7f5-4c08-4ccc-924c-5f8276f4eec2 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:22.934182 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 954/1891] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:22.992833 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 938/1892] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:22.995331 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71973c5-fffd-41c0-8a64-6b5956e9d768 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:22.996806 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71973c5-fffd-41c0-8a64-6b5956e9d768 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:22.997344 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71973c5-fffd-41c0-8a64-6b5956e9d768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:23.037281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f71973c5-fffd-41c0-8a64-6b5956e9d768 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:23.038634 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f71973c5-fffd-41c0-8a64-6b5956e9d768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:23.054805 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 955/1893] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:23.056447 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2fec4-b4b2-4965-b866-d6f384f1395f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:23.075594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2fec4-b4b2-4965-b866-d6f384f1395f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:23.089546 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4e0ee-cc13-4084-805c-ab6f6e5be6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:23.091161 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4e0ee-cc13-4084-805c-ab6f6e5be6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:23.091161 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4e0ee-cc13-4084-805c-ab6f6e5be6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:23.091161 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4e0ee-cc13-4084-805c-ab6f6e5be6a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:23.091596 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4e0ee-cc13-4084-805c-ab6f6e5be6a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:29:23.093985 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4e0ee-cc13-4084-805c-ab6f6e5be6a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:23.095801 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4e0ee-cc13-4084-805c-ab6f6e5be6a3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:23.118019 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 939/1894] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:29:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:29:23.123335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71973c5-fffd-41c0-8a64-6b5956e9d768 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:23.136272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71973c5-fffd-41c0-8a64-6b5956e9d768 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:23.143794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ffe9e-796a-4f98-9519-cb3a343f52a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:23.144662 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ffe9e-796a-4f98-9519-cb3a343f52a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:23.145062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ffe9e-796a-4f98-9519-cb3a343f52a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:23.145300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ffe9e-796a-4f98-9519-cb3a343f52a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:23.146050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ffe9e-796a-4f98-9519-cb3a343f52a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:23.148221 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ffe9e-796a-4f98-9519-cb3a343f52a5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:23.185441 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 956/1895] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:23 2022] POST /identity/v3/projects => generated 369 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:23.193120 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4e0ee-cc13-4084-805c-ab6f6e5be6a3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:23.200261 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4e0ee-cc13-4084-805c-ab6f6e5be6a3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:23.272581 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66a2c3-875f-4bda-8abe-da85ebf9e36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:23.273409 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66a2c3-875f-4bda-8abe-da85ebf9e36e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:23.273787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66a2c3-875f-4bda-8abe-da85ebf9e36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:23.274305 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66a2c3-875f-4bda-8abe-da85ebf9e36e admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:23.275052 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66a2c3-875f-4bda-8abe-da85ebf9e36e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:23.277941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66a2c3-875f-4bda-8abe-da85ebf9e36e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:23.299699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 940/1896] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:23 2022] POST /identity/v3/users => generated 285 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:23.304026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ffe9e-796a-4f98-9519-cb3a343f52a5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:23.317847 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ffe9e-796a-4f98-9519-cb3a343f52a5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:23.322560 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346cfb0-8933-4923-94a6-5e4b0cb1b7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:23.323416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346cfb0-8933-4923-94a6-5e4b0cb1b7ac admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:23.324747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346cfb0-8933-4923-94a6-5e4b0cb1b7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:23.325049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346cfb0-8933-4923-94a6-5e4b0cb1b7ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:23.327156 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346cfb0-8933-4923-94a6-5e4b0cb1b7ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:23.328256 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346cfb0-8933-4923-94a6-5e4b0cb1b7ac admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:23.339914 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 957/1897] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:23.344318 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66a2c3-875f-4bda-8abe-da85ebf9e36e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:23.355392 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66a2c3-875f-4bda-8abe-da85ebf9e36e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:23.360409 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507bbf6-9cec-4ece-9631-3c8c26ca5be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:23.361119 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507bbf6-9cec-4ece-9631-3c8c26ca5be1 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:23.361409 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507bbf6-9cec-4ece-9631-3c8c26ca5be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:23.361741 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507bbf6-9cec-4ece-9631-3c8c26ca5be1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:23.362479 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507bbf6-9cec-4ece-9631-3c8c26ca5be1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:23.364120 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507bbf6-9cec-4ece-9631-3c8c26ca5be1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:23.377899 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 941/1898] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:29:23.379188 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346cfb0-8933-4923-94a6-5e4b0cb1b7ac admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:23.391733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346cfb0-8933-4923-94a6-5e4b0cb1b7ac admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:23.398081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380ccca4-2ebf-47ae-a336-e6f733c28f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:23.398915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ccca4-2ebf-47ae-a336-e6f733c28f50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:23.399198 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ccca4-2ebf-47ae-a336-e6f733c28f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:23.399481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ccca4-2ebf-47ae-a336-e6f733c28f50 admin admin] PATH_INFO: `/v3/projects/92d388c9d937446d90a47eeb70eb54c4/users/44b95c5ee1374995b9d2d09cf662bb30/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:23.412602 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:23.428217 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380ccca4-2ebf-47ae-a336-e6f733c28f50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92d388c9d937446d90a47eeb70eb54c4, user_id=44b95c5ee1374995b9d2d09cf662bb30, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:23.431016 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380ccca4-2ebf-47ae-a336-e6f733c28f50 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:23.446424 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 958/1899] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:23 2022] PUT /identity/v3/projects/92d388c9d937446d90a47eeb70eb54c4/users/44b95c5ee1374995b9d2d09cf662bb30/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:23.453110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0dbca3-77e3-4e03-857c-37c436901873 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:23.453447 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0dbca3-77e3-4e03-857c-37c436901873 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:23.453838 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0dbca3-77e3-4e03-857c-37c436901873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:23.475696 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4a0dbca3-77e3-4e03-857c-37c436901873 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:23.476984 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4a0dbca3-77e3-4e03-857c-37c436901873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:23.539497 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:23.561619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 942/1900] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:23.570850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380ccca4-2ebf-47ae-a336-e6f733c28f50 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:23.574416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-380ccca4-2ebf-47ae-a336-e6f733c28f50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:23.638698 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0dbca3-77e3-4e03-857c-37c436901873 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:23.641415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4a0dbca3-77e3-4e03-857c-37c436901873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:23.645930 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380ccca4-2ebf-47ae-a336-e6f733c28f50 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:23.650547 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a647b-4d4b-42d4-984b-69e2689f4266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:23.651606 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a647b-4d4b-42d4-984b-69e2689f4266 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:23.652301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a647b-4d4b-42d4-984b-69e2689f4266 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:23.653168 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a647b-4d4b-42d4-984b-69e2689f4266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:23.660030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0dbca3-77e3-4e03-857c-37c436901873 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:23.665363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924d7a5-899b-4bf9-a136-77cc1522f500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:23.666344 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924d7a5-899b-4bf9-a136-77cc1522f500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:23.666634 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924d7a5-899b-4bf9-a136-77cc1522f500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:23.666918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924d7a5-899b-4bf9-a136-77cc1522f500 admin admin] PATH_INFO: `/v3/users/8bafad298e2247b399e0787600908794` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:23.668361 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924d7a5-899b-4bf9-a136-77cc1522f500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bafad298e2247b399e0787600908794)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:23.670081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a647b-4d4b-42d4-984b-69e2689f4266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:23.670459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924d7a5-899b-4bf9-a136-77cc1522f500 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:23.672251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a647b-4d4b-42d4-984b-69e2689f4266 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:23.687058 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 959/1901] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:23.728568 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1924d7a5-899b-4bf9-a136-77cc1522f500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bafad298e2247b399e0787600908794'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:23.729302 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1924d7a5-899b-4bf9-a136-77cc1522f500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bafad298e2247b399e0787600908794'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:23.743640 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1924d7a5-899b-4bf9-a136-77cc1522f500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bafad298e2247b399e0787600908794'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:23.753511 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1924d7a5-899b-4bf9-a136-77cc1522f500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bafad298e2247b399e0787600908794'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:23.754193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1924d7a5-899b-4bf9-a136-77cc1522f500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bafad298e2247b399e0787600908794'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:23.774119 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 943/1902] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:23 2022] DELETE /identity/v3/users/8bafad298e2247b399e0787600908794 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:23.778711 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48c08c-22d9-4183-a005-71ac9d96df32 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:23.779004 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48c08c-22d9-4183-a005-71ac9d96df32 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:23.779377 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48c08c-22d9-4183-a005-71ac9d96df32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:23.801239 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6e48c08c-22d9-4183-a005-71ac9d96df32 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:23.802807 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6e48c08c-22d9-4183-a005-71ac9d96df32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:23.881682 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 960/1903] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:23.892004 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924d7a5-899b-4bf9-a136-77cc1522f500 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:23.893457 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1924d7a5-899b-4bf9-a136-77cc1522f500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:23.974427 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924d7a5-899b-4bf9-a136-77cc1522f500 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:23.978349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d1cd6-7eab-4846-84cb-15b8a97f0bec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:23.978988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d1cd6-7eab-4846-84cb-15b8a97f0bec service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:23.979277 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d1cd6-7eab-4846-84cb-15b8a97f0bec service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:23.979567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d1cd6-7eab-4846-84cb-15b8a97f0bec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:23.991454 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d1cd6-7eab-4846-84cb-15b8a97f0bec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:23.993600 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d1cd6-7eab-4846-84cb-15b8a97f0bec service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:24.006982 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 944/1904] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:24.403781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48c08c-22d9-4183-a005-71ac9d96df32 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:24.406458 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6e48c08c-22d9-4183-a005-71ac9d96df32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:24.444734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48c08c-22d9-4183-a005-71ac9d96df32 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:24.455694 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297f7fe-7b81-4026-aeea-5a98e48c199b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:24.456783 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297f7fe-7b81-4026-aeea-5a98e48c199b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:24.457414 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297f7fe-7b81-4026-aeea-5a98e48c199b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:24.459298 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297f7fe-7b81-4026-aeea-5a98e48c199b admin admin] PATH_INFO: `/v3/projects/2b032dfca7f44f13a41d06ff5346734e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:24.461453 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297f7fe-7b81-4026-aeea-5a98e48c199b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b032dfca7f44f13a41d06ff5346734e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:24.464036 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297f7fe-7b81-4026-aeea-5a98e48c199b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:24.536909 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b297f7fe-7b81-4026-aeea-5a98e48c199b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b032dfca7f44f13a41d06ff5346734e', 'request_id': 'req-b297f7fe-7b81-4026-aeea-5a98e48c199b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:24.546336 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b297f7fe-7b81-4026-aeea-5a98e48c199b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b032dfca7f44f13a41d06ff5346734e', 'request_id': 'req-b297f7fe-7b81-4026-aeea-5a98e48c199b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:24.556209 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b297f7fe-7b81-4026-aeea-5a98e48c199b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b032dfca7f44f13a41d06ff5346734e', 'request_id': 'req-b297f7fe-7b81-4026-aeea-5a98e48c199b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:24.574630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b297f7fe-7b81-4026-aeea-5a98e48c199b admin admin] The token cache is being invalidate because project 2b032dfca7f44f13a41d06ff5346734e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:24.574972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b297f7fe-7b81-4026-aeea-5a98e48c199b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:24.575768 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b297f7fe-7b81-4026-aeea-5a98e48c199b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:24.579301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 961/1905] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:24 2022] DELETE /identity/v3/projects/2b032dfca7f44f13a41d06ff5346734e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:24.583626 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d1cd6-7eab-4846-84cb-15b8a97f0bec service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:24.595243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d1cd6-7eab-4846-84cb-15b8a97f0bec service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:24.694433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8808ee1b-ed22-46ad-8fc7-e09f0d72b7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:24.695288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808ee1b-ed22-46ad-8fc7-e09f0d72b7d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:24.695572 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808ee1b-ed22-46ad-8fc7-e09f0d72b7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:24.695847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808ee1b-ed22-46ad-8fc7-e09f0d72b7d5 admin admin] PATH_INFO: `/v3/users/0cb606739bfb4c78aa9dac40cecc26ae` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:24.698624 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8808ee1b-ed22-46ad-8fc7-e09f0d72b7d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cb606739bfb4c78aa9dac40cecc26ae)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:24.700719 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8808ee1b-ed22-46ad-8fc7-e09f0d72b7d5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:24.707440 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:24.752639 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8808ee1b-ed22-46ad-8fc7-e09f0d72b7d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cb606739bfb4c78aa9dac40cecc26ae'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:24.753540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8808ee1b-ed22-46ad-8fc7-e09f0d72b7d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cb606739bfb4c78aa9dac40cecc26ae'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:24.766114 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8808ee1b-ed22-46ad-8fc7-e09f0d72b7d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cb606739bfb4c78aa9dac40cecc26ae'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:24.778222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8808ee1b-ed22-46ad-8fc7-e09f0d72b7d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cb606739bfb4c78aa9dac40cecc26ae'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:24.778884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8808ee1b-ed22-46ad-8fc7-e09f0d72b7d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cb606739bfb4c78aa9dac40cecc26ae'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:24.791978 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 945/1906] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:24 2022] DELETE /identity/v3/users/0cb606739bfb4c78aa9dac40cecc26ae => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:24.881189 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7e004-a620-45bc-824b-ec81a7979733 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:24.881969 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7e004-a620-45bc-824b-ec81a7979733 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:24.882559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7e004-a620-45bc-824b-ec81a7979733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:24.912784 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:24.915201 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5ff7e004-a620-45bc-824b-ec81a7979733 None None] MFA Rules not processed for user `44b95c5ee1374995b9d2d09cf662bb30`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:24.917359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5ff7e004-a620-45bc-824b-ec81a7979733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:25.002558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 962/1907] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:24 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:25.007049 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8808ee1b-ed22-46ad-8fc7-e09f0d72b7d5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:25.008850 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8808ee1b-ed22-46ad-8fc7-e09f0d72b7d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:25.120649 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8808ee1b-ed22-46ad-8fc7-e09f0d72b7d5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:25.127587 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e71b1-9082-42aa-8dac-d9854ffcd195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:25.128283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e71b1-9082-42aa-8dac-d9854ffcd195 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:25.128571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e71b1-9082-42aa-8dac-d9854ffcd195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:25.128849 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e71b1-9082-42aa-8dac-d9854ffcd195 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:25.129547 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e71b1-9082-42aa-8dac-d9854ffcd195 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:25.131576 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e71b1-9082-42aa-8dac-d9854ffcd195 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:25.164089 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 946/1908] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:25 2022] POST /identity/v3/projects => generated 371 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:25.167846 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7e004-a620-45bc-824b-ec81a7979733 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:25.180640 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7e004-a620-45bc-824b-ec81a7979733 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:25.225948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e71b1-9082-42aa-8dac-d9854ffcd195 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:25.227990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-673e71b1-9082-42aa-8dac-d9854ffcd195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:25.257317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099dc97-9363-4436-9ca8-12ef3a046a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:25.257941 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099dc97-9363-4436-9ca8-12ef3a046a25 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:25.258227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099dc97-9363-4436-9ca8-12ef3a046a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:25.258508 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099dc97-9363-4436-9ca8-12ef3a046a25 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:25.259203 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3099dc97-9363-4436-9ca8-12ef3a046a25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:25.261136 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3099dc97-9363-4436-9ca8-12ef3a046a25 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:25.285737 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 963/1909] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:25 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:25.290350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099dc97-9363-4436-9ca8-12ef3a046a25 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:25.308205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099dc97-9363-4436-9ca8-12ef3a046a25 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:25.315159 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c8d10-fe94-4862-b129-720f75c36b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:25.315793 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e71b1-9082-42aa-8dac-d9854ffcd195 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:25.315940 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c8d10-fe94-4862-b129-720f75c36b1c admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:25.317015 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c8d10-fe94-4862-b129-720f75c36b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:25.317015 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c8d10-fe94-4862-b129-720f75c36b1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:25.317186 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c8d10-fe94-4862-b129-720f75c36b1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:25.319910 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c8d10-fe94-4862-b129-720f75c36b1c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:25.332267 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2485a4-764f-49b7-b391-73a298a051bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:25.334062 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2485a4-764f-49b7-b391-73a298a051bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:25.334352 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2485a4-764f-49b7-b391-73a298a051bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:25.334635 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2485a4-764f-49b7-b391-73a298a051bf admin admin] PATH_INFO: `/v3/projects/433968dd9da34d889fbfaa517bf22e09` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:25.336014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2485a4-764f-49b7-b391-73a298a051bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=433968dd9da34d889fbfaa517bf22e09)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:25.336568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 964/1910] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:29:25.338130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2485a4-764f-49b7-b391-73a298a051bf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:25.343132 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c8d10-fe94-4862-b129-720f75c36b1c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:25.359333 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c8d10-fe94-4862-b129-720f75c36b1c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:25.362116 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:25.366108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36d08a-4dc4-47ac-9b3b-86b58c42243e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:25.367334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36d08a-4dc4-47ac-9b3b-86b58c42243e admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:25.367569 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36d08a-4dc4-47ac-9b3b-86b58c42243e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:25.367857 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36d08a-4dc4-47ac-9b3b-86b58c42243e admin admin] PATH_INFO: `/v3/projects/73706b4b9b3441b6ad1cb77ec25f3e71/users/7bff3b7c7bb84e5e8baaca8e23450b28/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:25.387061 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36d08a-4dc4-47ac-9b3b-86b58c42243e admin admin] RBAC: Authorizing `identity:create_grant(project_id=73706b4b9b3441b6ad1cb77ec25f3e71, user_id=7bff3b7c7bb84e5e8baaca8e23450b28, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:25.391008 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36d08a-4dc4-47ac-9b3b-86b58c42243e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:25.407746 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 965/1911] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:25 2022] PUT /identity/v3/projects/73706b4b9b3441b6ad1cb77ec25f3e71/users/7bff3b7c7bb84e5e8baaca8e23450b28/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:25.412963 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36d08a-4dc4-47ac-9b3b-86b58c42243e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:25.424282 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-af2485a4-764f-49b7-b391-73a298a051bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '433968dd9da34d889fbfaa517bf22e09', 'request_id': 'req-af2485a4-764f-49b7-b391-73a298a051bf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:25.431877 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-af2485a4-764f-49b7-b391-73a298a051bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '433968dd9da34d889fbfaa517bf22e09', 'request_id': 'req-af2485a4-764f-49b7-b391-73a298a051bf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:25.439005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-af2485a4-764f-49b7-b391-73a298a051bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '433968dd9da34d889fbfaa517bf22e09', 'request_id': 'req-af2485a4-764f-49b7-b391-73a298a051bf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:25.454851 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-af2485a4-764f-49b7-b391-73a298a051bf admin admin] The token cache is being invalidate because project 433968dd9da34d889fbfaa517bf22e09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:25.455162 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-af2485a4-764f-49b7-b391-73a298a051bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:25.456167 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-af2485a4-764f-49b7-b391-73a298a051bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:25.458513 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 947/1912] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:25 2022] DELETE /identity/v3/projects/433968dd9da34d889fbfaa517bf22e09 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:25.464528 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ddf60-a857-474a-9b37-4191467ea8c5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:25.464812 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ddf60-a857-474a-9b37-4191467ea8c5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:25.466134 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ddf60-a857-474a-9b37-4191467ea8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:25.503751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:25.508391 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36d08a-4dc4-47ac-9b3b-86b58c42243e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:25.604787 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd03b35-91c8-4ac6-af0e-89f20fb16fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:25.606361 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd03b35-91c8-4ac6-af0e-89f20fb16fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:25.607027 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd03b35-91c8-4ac6-af0e-89f20fb16fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:25.608498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd03b35-91c8-4ac6-af0e-89f20fb16fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:25.609858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd03b35-91c8-4ac6-af0e-89f20fb16fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:25.612528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd03b35-91c8-4ac6-af0e-89f20fb16fc3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:25.631615 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 966/1913] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:25.631615 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd03b35-91c8-4ac6-af0e-89f20fb16fc3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:25.633522 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5fd03b35-91c8-4ac6-af0e-89f20fb16fc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:25.648207 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd03b35-91c8-4ac6-af0e-89f20fb16fc3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:25.658417 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-149ddf60-a857-474a-9b37-4191467ea8c5 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:25.660321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-149ddf60-a857-474a-9b37-4191467ea8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:25.670836 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 948/1914] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:25.673715 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c59346-8d13-43c7-81ef-5eed715091e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:25.674815 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c59346-8d13-43c7-81ef-5eed715091e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:25.675209 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c59346-8d13-43c7-81ef-5eed715091e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:25.677739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c59346-8d13-43c7-81ef-5eed715091e9 admin admin] PATH_INFO: `/v3/projects/73706b4b9b3441b6ad1cb77ec25f3e71/users/7bff3b7c7bb84e5e8baaca8e23450b28/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:25.679060 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a7ddd-4d0c-48ae-8029-b7c9443f0193 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:25.679535 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a7ddd-4d0c-48ae-8029-b7c9443f0193 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:25.679821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a7ddd-4d0c-48ae-8029-b7c9443f0193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:25.690021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c59346-8d13-43c7-81ef-5eed715091e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73706b4b9b3441b6ad1cb77ec25f3e71, user_id=7bff3b7c7bb84e5e8baaca8e23450b28, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:25.693359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c59346-8d13-43c7-81ef-5eed715091e9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:25.708973 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 967/1915] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:25 2022] PUT /identity/v3/projects/73706b4b9b3441b6ad1cb77ec25f3e71/users/7bff3b7c7bb84e5e8baaca8e23450b28/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:25.719424 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c59346-8d13-43c7-81ef-5eed715091e9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:25.723113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-08c59346-8d13-43c7-81ef-5eed715091e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:25.726685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-377a7ddd-4d0c-48ae-8029-b7c9443f0193 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:25.728208 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-377a7ddd-4d0c-48ae-8029-b7c9443f0193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:25.834334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c59346-8d13-43c7-81ef-5eed715091e9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:25.841104 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776e121-5035-4da2-90ec-138824e3a793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:25.843127 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776e121-5035-4da2-90ec-138824e3a793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:25.843283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776e121-5035-4da2-90ec-138824e3a793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:25.843652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776e121-5035-4da2-90ec-138824e3a793 admin admin] PATH_INFO: `/v3/users/e146fbe0451c44389e57276b4af5a55e` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:25.845302 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776e121-5035-4da2-90ec-138824e3a793 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e146fbe0451c44389e57276b4af5a55e)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:25.847826 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 949/1916] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:29:25.848372 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776e121-5035-4da2-90ec-138824e3a793 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:25.853004 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a7ddd-4d0c-48ae-8029-b7c9443f0193 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:25.866342 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a7ddd-4d0c-48ae-8029-b7c9443f0193 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:25.880645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700fb67c-3210-42a5-9baf-90ee0cf8f6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:25.881658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fb67c-3210-42a5-9baf-90ee0cf8f6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:25.885938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fb67c-3210-42a5-9baf-90ee0cf8f6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:25.886738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fb67c-3210-42a5-9baf-90ee0cf8f6a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:25.890858 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700fb67c-3210-42a5-9baf-90ee0cf8f6a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:29:25.891604 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700fb67c-3210-42a5-9baf-90ee0cf8f6a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:25.894865 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700fb67c-3210-42a5-9baf-90ee0cf8f6a3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:25.911767 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 950/1917] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:29:25.916299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700fb67c-3210-42a5-9baf-90ee0cf8f6a3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:25.927480 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700fb67c-3210-42a5-9baf-90ee0cf8f6a3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:25.928926 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3776e121-5035-4da2-90ec-138824e3a793 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e146fbe0451c44389e57276b4af5a55e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:25.933105 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39eff1-72d8-4d7e-b223-290f0d32b420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:25.933813 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39eff1-72d8-4d7e-b223-290f0d32b420 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:25.934100 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39eff1-72d8-4d7e-b223-290f0d32b420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:25.934382 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39eff1-72d8-4d7e-b223-290f0d32b420 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:25.935454 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39eff1-72d8-4d7e-b223-290f0d32b420 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:25.938578 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39eff1-72d8-4d7e-b223-290f0d32b420 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:25.953318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3776e121-5035-4da2-90ec-138824e3a793 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e146fbe0451c44389e57276b4af5a55e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:25.954844 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3776e121-5035-4da2-90ec-138824e3a793 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e146fbe0451c44389e57276b4af5a55e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:25.970478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3776e121-5035-4da2-90ec-138824e3a793 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e146fbe0451c44389e57276b4af5a55e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:25.971339 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3776e121-5035-4da2-90ec-138824e3a793 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e146fbe0451c44389e57276b4af5a55e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:26.006298 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 951/1918] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:25 2022] POST /identity/v3/projects => generated 381 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:29:26.011415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39eff1-72d8-4d7e-b223-290f0d32b420 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:26.015465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ce39eff1-72d8-4d7e-b223-290f0d32b420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:26.017871 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 968/1919] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:25 2022] DELETE /identity/v3/users/e146fbe0451c44389e57276b4af5a55e => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:26.022458 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979ea28-89ca-4e47-bd2d-5d65d963e894 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:26.022915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979ea28-89ca-4e47-bd2d-5d65d963e894 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:26.023430 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979ea28-89ca-4e47-bd2d-5d65d963e894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:26.087004 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:26.108875 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2979ea28-89ca-4e47-bd2d-5d65d963e894 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:26.110265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2979ea28-89ca-4e47-bd2d-5d65d963e894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:26.120771 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39eff1-72d8-4d7e-b223-290f0d32b420 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:26.129205 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:26.218514 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a02991-e83b-41e3-adfb-f18f6d1f0b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:26.219646 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a02991-e83b-41e3-adfb-f18f6d1f0b7e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:26.219939 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a02991-e83b-41e3-adfb-f18f6d1f0b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:26.220219 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a02991-e83b-41e3-adfb-f18f6d1f0b7e admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:26.220990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a02991-e83b-41e3-adfb-f18f6d1f0b7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:26.223596 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a02991-e83b-41e3-adfb-f18f6d1f0b7e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:26.229665 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 969/1920] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:26.238904 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979ea28-89ca-4e47-bd2d-5d65d963e894 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:26.240806 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2979ea28-89ca-4e47-bd2d-5d65d963e894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:26.253736 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 952/1921] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:26 2022] POST /identity/v3/users => generated 291 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:26.258090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a02991-e83b-41e3-adfb-f18f6d1f0b7e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:26.282975 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a02991-e83b-41e3-adfb-f18f6d1f0b7e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:26.288482 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aafd559-7b9a-4d99-b8f2-ad7174706fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:26.289811 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafd559-7b9a-4d99-b8f2-ad7174706fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:26.290177 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafd559-7b9a-4d99-b8f2-ad7174706fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:26.290537 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafd559-7b9a-4d99-b8f2-ad7174706fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:26.291869 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aafd559-7b9a-4d99-b8f2-ad7174706fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:26.293869 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aafd559-7b9a-4d99-b8f2-ad7174706fb0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:26.308864 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 953/1922] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:29:26.314212 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aafd559-7b9a-4d99-b8f2-ad7174706fb0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:26.329034 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aafd559-7b9a-4d99-b8f2-ad7174706fb0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:26.337630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb94d11-aa79-40bf-af80-6d25431d12cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:26.338300 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb94d11-aa79-40bf-af80-6d25431d12cc admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:26.338587 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb94d11-aa79-40bf-af80-6d25431d12cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:26.338869 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb94d11-aa79-40bf-af80-6d25431d12cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:26.339609 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb94d11-aa79-40bf-af80-6d25431d12cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:26.341030 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979ea28-89ca-4e47-bd2d-5d65d963e894 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:26.341221 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb94d11-aa79-40bf-af80-6d25431d12cc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:26.348475 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc54fc-2e0a-4c4a-bd4d-b754460269a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:26.349138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc54fc-2e0a-4c4a-bd4d-b754460269a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:26.349557 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc54fc-2e0a-4c4a-bd4d-b754460269a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:26.350653 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc54fc-2e0a-4c4a-bd4d-b754460269a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:26.357767 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 954/1923] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:29:26.361279 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb94d11-aa79-40bf-af80-6d25431d12cc admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:26.365483 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc54fc-2e0a-4c4a-bd4d-b754460269a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:26.367964 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc54fc-2e0a-4c4a-bd4d-b754460269a2 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:26.380353 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb94d11-aa79-40bf-af80-6d25431d12cc admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:26.386566 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b01e06-b222-4866-9243-fae4c62b630c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:26.387870 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b01e06-b222-4866-9243-fae4c62b630c admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:26.388370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b01e06-b222-4866-9243-fae4c62b630c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:26.390307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b01e06-b222-4866-9243-fae4c62b630c admin admin] PATH_INFO: `/v3/projects/a281a7fa5a6947ab9b181d7e190ac68a/users/3440036e801f410fa6734744304db483/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:26.391919 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 970/1924] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:26.415081 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b01e06-b222-4866-9243-fae4c62b630c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a281a7fa5a6947ab9b181d7e190ac68a, user_id=3440036e801f410fa6734744304db483, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:26.419065 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b01e06-b222-4866-9243-fae4c62b630c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:26.447036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 955/1925] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:26 2022] PUT /identity/v3/projects/a281a7fa5a6947ab9b181d7e190ac68a/users/3440036e801f410fa6734744304db483/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:26.459434 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a7327-7302-45a3-8707-ef75e17a581b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:26.459988 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a7327-7302-45a3-8707-ef75e17a581b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:26.460087 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a7327-7302-45a3-8707-ef75e17a581b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:26.554417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0f2a7327-7302-45a3-8707-ef75e17a581b None None] MFA Rules not processed for user `3440036e801f410fa6734744304db483`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:26.555687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0f2a7327-7302-45a3-8707-ef75e17a581b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:26.665918 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 971/1926] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:26.672065 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b01e06-b222-4866-9243-fae4c62b630c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:26.742523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b01e06-b222-4866-9243-fae4c62b630c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:26.749176 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed47221-4dc0-4702-865e-6ba0d7b9d13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:26.750176 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed47221-4dc0-4702-865e-6ba0d7b9d13c admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:26.750176 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed47221-4dc0-4702-865e-6ba0d7b9d13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:26.750487 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed47221-4dc0-4702-865e-6ba0d7b9d13c admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:26.751301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed47221-4dc0-4702-865e-6ba0d7b9d13c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:26.753282 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed47221-4dc0-4702-865e-6ba0d7b9d13c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:26.778082 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a7327-7302-45a3-8707-ef75e17a581b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:26.780227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0f2a7327-7302-45a3-8707-ef75e17a581b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:26.796388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a7327-7302-45a3-8707-ef75e17a581b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:26.802196 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a25bf7-4bc3-40d2-a2df-991380ba5368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:26.803147 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a25bf7-4bc3-40d2-a2df-991380ba5368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:26.803441 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a25bf7-4bc3-40d2-a2df-991380ba5368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:26.803726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a25bf7-4bc3-40d2-a2df-991380ba5368 admin admin] PATH_INFO: `/v3/projects/8670df9b372441429766eb047c891e92` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:26.804942 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a25bf7-4bc3-40d2-a2df-991380ba5368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8670df9b372441429766eb047c891e92)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:26.807011 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a25bf7-4bc3-40d2-a2df-991380ba5368 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:26.813394 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 956/1927] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:26 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:26.814477 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed47221-4dc0-4702-865e-6ba0d7b9d13c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:26.894804 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:26.919201 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b1a25bf7-4bc3-40d2-a2df-991380ba5368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8670df9b372441429766eb047c891e92', 'request_id': 'req-b1a25bf7-4bc3-40d2-a2df-991380ba5368'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:26.928460 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:26.938233 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b1a25bf7-4bc3-40d2-a2df-991380ba5368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8670df9b372441429766eb047c891e92', 'request_id': 'req-b1a25bf7-4bc3-40d2-a2df-991380ba5368'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:26.948412 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b1a25bf7-4bc3-40d2-a2df-991380ba5368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8670df9b372441429766eb047c891e92', 'request_id': 'req-b1a25bf7-4bc3-40d2-a2df-991380ba5368'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:26.970891 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b1a25bf7-4bc3-40d2-a2df-991380ba5368 admin admin] The token cache is being invalidate because project 8670df9b372441429766eb047c891e92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:26.971193 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b1a25bf7-4bc3-40d2-a2df-991380ba5368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:26.971992 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b1a25bf7-4bc3-40d2-a2df-991380ba5368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:26.974506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed47221-4dc0-4702-865e-6ba0d7b9d13c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:26.978004 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 972/1928] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:26 2022] DELETE /identity/v3/projects/8670df9b372441429766eb047c891e92 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:26.980078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1356d-3b5d-415d-8c8e-62c03d44d65c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:26.982131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1356d-3b5d-415d-8c8e-62c03d44d65c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:26.983147 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1356d-3b5d-415d-8c8e-62c03d44d65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:27.019700 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-daa1356d-3b5d-415d-8c8e-62c03d44d65c None None] MFA Rules not processed for user `7bff3b7c7bb84e5e8baaca8e23450b28`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:27.026353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-daa1356d-3b5d-415d-8c8e-62c03d44d65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:27.079004 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed3811-a344-4824-b814-b911733744c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:27.080024 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed3811-a344-4824-b814-b911733744c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:27.080557 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed3811-a344-4824-b814-b911733744c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:27.080921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed3811-a344-4824-b814-b911733744c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:27.081866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed3811-a344-4824-b814-b911733744c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:27.084265 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed3811-a344-4824-b814-b911733744c8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:27.116451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 957/1929] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:26 2022] POST /identity/v3/users => generated 296 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:27.116451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed3811-a344-4824-b814-b911733744c8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:27.116451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-48ed3811-a344-4824-b814-b911733744c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:27.130935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 973/1930] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:26 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:27.131760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed3811-a344-4824-b814-b911733744c8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:27.132327 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1356d-3b5d-415d-8c8e-62c03d44d65c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:27.134431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-daa1356d-3b5d-415d-8c8e-62c03d44d65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:27.137192 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14777e74-fa39-4fdf-bc23-14b4768f7eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:27.138220 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14777e74-fa39-4fdf-bc23-14b4768f7eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:27.138569 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14777e74-fa39-4fdf-bc23-14b4768f7eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:27.139418 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14777e74-fa39-4fdf-bc23-14b4768f7eba admin admin] PATH_INFO: `/v3/users/f510a5006aa144bbaafb33feea40f83b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:27.141227 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14777e74-fa39-4fdf-bc23-14b4768f7eba admin admin] RBAC: Authorizing `identity:delete_user(user_id=f510a5006aa144bbaafb33feea40f83b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:27.143787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14777e74-fa39-4fdf-bc23-14b4768f7eba admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:27.160552 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1356d-3b5d-415d-8c8e-62c03d44d65c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:27.165901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d94487-90ea-476c-8369-8bb59bad534d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:27.166437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d94487-90ea-476c-8369-8bb59bad534d admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:27.166774 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d94487-90ea-476c-8369-8bb59bad534d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:27.167105 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d94487-90ea-476c-8369-8bb59bad534d admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:27.167843 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d94487-90ea-476c-8369-8bb59bad534d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:27.169595 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d94487-90ea-476c-8369-8bb59bad534d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:27.181873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 974/1931] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:27.182631 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d94487-90ea-476c-8369-8bb59bad534d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:27.187442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-90d94487-90ea-476c-8369-8bb59bad534d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:27.228206 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-14777e74-fa39-4fdf-bc23-14b4768f7eba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f510a5006aa144bbaafb33feea40f83b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:27.229165 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-14777e74-fa39-4fdf-bc23-14b4768f7eba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f510a5006aa144bbaafb33feea40f83b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:27.247132 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-14777e74-fa39-4fdf-bc23-14b4768f7eba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f510a5006aa144bbaafb33feea40f83b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:27.258492 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-14777e74-fa39-4fdf-bc23-14b4768f7eba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f510a5006aa144bbaafb33feea40f83b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:27.259798 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d94487-90ea-476c-8369-8bb59bad534d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:27.260021 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-14777e74-fa39-4fdf-bc23-14b4768f7eba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f510a5006aa144bbaafb33feea40f83b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:27.265604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab33b5-edbd-420f-b191-10c89a537100 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:27.266533 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab33b5-edbd-420f-b191-10c89a537100 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:27.266898 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab33b5-edbd-420f-b191-10c89a537100 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:27.267280 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab33b5-edbd-420f-b191-10c89a537100 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:27.270496 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-daab33b5-edbd-420f-b191-10c89a537100 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:27.278123 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 958/1932] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:27 2022] DELETE /identity/v3/users/f510a5006aa144bbaafb33feea40f83b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:27.279069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14777e74-fa39-4fdf-bc23-14b4768f7eba admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:27.281186 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-14777e74-fa39-4fdf-bc23-14b4768f7eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:27.362317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:27.368457 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:27.380358 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab33b5-edbd-420f-b191-10c89a537100 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:27.382571 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab33b5-edbd-420f-b191-10c89a537100 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:27.396713 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 975/1933] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:27 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:27.414495 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14777e74-fa39-4fdf-bc23-14b4768f7eba admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:27.421522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3a131-11f7-41c3-92ec-d3cd68147ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:27.422706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3a131-11f7-41c3-92ec-d3cd68147ae3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:27.423058 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3a131-11f7-41c3-92ec-d3cd68147ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:27.423363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3a131-11f7-41c3-92ec-d3cd68147ae3 admin admin] PATH_INFO: `/v3/projects/56070ec453e84f49bf315c969f346f87/users/f6dab8c66d6241e68ac7c88b077afbbb/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:27.443333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3a131-11f7-41c3-92ec-d3cd68147ae3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56070ec453e84f49bf315c969f346f87, user_id=f6dab8c66d6241e68ac7c88b077afbbb, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:27.447607 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3a131-11f7-41c3-92ec-d3cd68147ae3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:27.471094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 959/1934] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:27 2022] PUT /identity/v3/projects/56070ec453e84f49bf315c969f346f87/users/f6dab8c66d6241e68ac7c88b077afbbb/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:27.475954 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab33b5-edbd-420f-b191-10c89a537100 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:27.491134 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab33b5-edbd-420f-b191-10c89a537100 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:27.637204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5ed55-1d3f-41f4-8d06-d38356145806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:27.638019 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5ed55-1d3f-41f4-8d06-d38356145806 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:27.638348 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5ed55-1d3f-41f4-8d06-d38356145806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:27.648686 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5ed55-1d3f-41f4-8d06-d38356145806 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:27.649807 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5ed55-1d3f-41f4-8d06-d38356145806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:27.652012 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5ed55-1d3f-41f4-8d06-d38356145806 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:27.677251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 976/1935] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:27.680317 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3a131-11f7-41c3-92ec-d3cd68147ae3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:27.693367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3a131-11f7-41c3-92ec-d3cd68147ae3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:27.699683 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbded2cc-22ac-44f8-bb80-c43e282b5822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:27.700399 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbded2cc-22ac-44f8-bb80-c43e282b5822 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:27.700688 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbded2cc-22ac-44f8-bb80-c43e282b5822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:27.700976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbded2cc-22ac-44f8-bb80-c43e282b5822 admin admin] PATH_INFO: `/v3/projects/56070ec453e84f49bf315c969f346f87/users/f6dab8c66d6241e68ac7c88b077afbbb/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:27.704651 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbded2cc-22ac-44f8-bb80-c43e282b5822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56070ec453e84f49bf315c969f346f87, user_id=f6dab8c66d6241e68ac7c88b077afbbb, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:27.708074 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbded2cc-22ac-44f8-bb80-c43e282b5822 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:27.721618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 960/1936] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:27 2022] PUT /identity/v3/projects/56070ec453e84f49bf315c969f346f87/users/f6dab8c66d6241e68ac7c88b077afbbb/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:27.731598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a230c96-9725-4602-8b5d-52ad17578c20 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:27.731897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a230c96-9725-4602-8b5d-52ad17578c20 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:27.732219 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a230c96-9725-4602-8b5d-52ad17578c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:27.746830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbded2cc-22ac-44f8-bb80-c43e282b5822 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:27.750857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bbded2cc-22ac-44f8-bb80-c43e282b5822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:27.774918 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4a230c96-9725-4602-8b5d-52ad17578c20 None None] MFA Rules not processed for user `f6dab8c66d6241e68ac7c88b077afbbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:27.777989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4a230c96-9725-4602-8b5d-52ad17578c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:27.836322 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbded2cc-22ac-44f8-bb80-c43e282b5822 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:27.849629 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99005927-ead1-42c9-aa56-9e209ad53cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:27.867590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99005927-ead1-42c9-aa56-9e209ad53cce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:27.867975 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99005927-ead1-42c9-aa56-9e209ad53cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:27.868174 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99005927-ead1-42c9-aa56-9e209ad53cce admin admin] PATH_INFO: `/v3/projects/9dd6d8b27c1b4d3f9a2fae94f516ed24` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:27.872473 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99005927-ead1-42c9-aa56-9e209ad53cce admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dd6d8b27c1b4d3f9a2fae94f516ed24)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:27.876157 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99005927-ead1-42c9-aa56-9e209ad53cce admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:27.900241 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 977/1937] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:27.913594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a230c96-9725-4602-8b5d-52ad17578c20 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:27.915000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4a230c96-9725-4602-8b5d-52ad17578c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:27.960464 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-99005927-ead1-42c9-aa56-9e209ad53cce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dd6d8b27c1b4d3f9a2fae94f516ed24', 'request_id': 'req-99005927-ead1-42c9-aa56-9e209ad53cce'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:27.971943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-99005927-ead1-42c9-aa56-9e209ad53cce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dd6d8b27c1b4d3f9a2fae94f516ed24', 'request_id': 'req-99005927-ead1-42c9-aa56-9e209ad53cce'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:27.985657 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-99005927-ead1-42c9-aa56-9e209ad53cce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dd6d8b27c1b4d3f9a2fae94f516ed24', 'request_id': 'req-99005927-ead1-42c9-aa56-9e209ad53cce'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:28.010127 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a230c96-9725-4602-8b5d-52ad17578c20 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:28.033872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-99005927-ead1-42c9-aa56-9e209ad53cce admin admin] The token cache is being invalidate because project 9dd6d8b27c1b4d3f9a2fae94f516ed24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:28.034712 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-99005927-ead1-42c9-aa56-9e209ad53cce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:28.036329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-99005927-ead1-42c9-aa56-9e209ad53cce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:28.041429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 961/1938] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:27 2022] DELETE /identity/v3/projects/9dd6d8b27c1b4d3f9a2fae94f516ed24 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:28.047382 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e30f88-da34-4ded-afea-742746c6b0c7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:28.047672 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e30f88-da34-4ded-afea-742746c6b0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:28.047955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e30f88-da34-4ded-afea-742746c6b0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:28.053538 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:28.071360 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-56e30f88-da34-4ded-afea-742746c6b0c7 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:28.072767 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-56e30f88-da34-4ded-afea-742746c6b0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:28.091069 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c42bc-cbbe-42e9-bdcc-e808f234ff3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:28.092112 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c42bc-cbbe-42e9-bdcc-e808f234ff3e service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:28.092578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c42bc-cbbe-42e9-bdcc-e808f234ff3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:28.093033 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c42bc-cbbe-42e9-bdcc-e808f234ff3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:28.095510 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fb2c42bc-cbbe-42e9-bdcc-e808f234ff3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:28.118260 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:28.149213 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 962/1939] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:28.164244 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88033857-e96e-4c58-bdfd-25af84cfc4a0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:28.164687 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88033857-e96e-4c58-bdfd-25af84cfc4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:28.164973 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88033857-e96e-4c58-bdfd-25af84cfc4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:28.188512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-88033857-e96e-4c58-bdfd-25af84cfc4a0 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:28.189423 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c42bc-cbbe-42e9-bdcc-e808f234ff3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:28.190351 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-88033857-e96e-4c58-bdfd-25af84cfc4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:28.190930 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c42bc-cbbe-42e9-bdcc-e808f234ff3e service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:28.198944 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 963/1940] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:29:28.203194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88033857-e96e-4c58-bdfd-25af84cfc4a0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:28.203960 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 978/1941] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:28.212658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88033857-e96e-4c58-bdfd-25af84cfc4a0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:28.218384 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc65be-583a-4b94-ac28-d4d8fab8f36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:28.219050 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc65be-583a-4b94-ac28-d4d8fab8f36f admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:28.219337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc65be-583a-4b94-ac28-d4d8fab8f36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:28.219618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc65be-583a-4b94-ac28-d4d8fab8f36f admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:28.220353 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc65be-583a-4b94-ac28-d4d8fab8f36f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:29:28.220695 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc65be-583a-4b94-ac28-d4d8fab8f36f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:28.222365 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc65be-583a-4b94-ac28-d4d8fab8f36f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:28.241030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 964/1942] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:29:28.243560 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c42bc-cbbe-42e9-bdcc-e808f234ff3e service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:28.257427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c42bc-cbbe-42e9-bdcc-e808f234ff3e service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:28.267367 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8871c4f-45c4-470d-b542-f608f94de147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:28.268045 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8871c4f-45c4-470d-b542-f608f94de147 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:28.268337 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8871c4f-45c4-470d-b542-f608f94de147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:28.268625 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8871c4f-45c4-470d-b542-f608f94de147 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:28.269409 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8871c4f-45c4-470d-b542-f608f94de147 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:28.271688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8871c4f-45c4-470d-b542-f608f94de147 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:28.302181 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 979/1943] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:28 2022] POST /identity/v3/projects => generated 369 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:28.307091 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc65be-583a-4b94-ac28-d4d8fab8f36f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:28.323497 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc65be-583a-4b94-ac28-d4d8fab8f36f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:28.421732 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91aa67-6911-4d4f-8e67-069d7acf6fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:28.421732 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91aa67-6911-4d4f-8e67-069d7acf6fdb admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:28.421732 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91aa67-6911-4d4f-8e67-069d7acf6fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:28.421732 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91aa67-6911-4d4f-8e67-069d7acf6fdb admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:28.422043 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91aa67-6911-4d4f-8e67-069d7acf6fdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:28.425400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91aa67-6911-4d4f-8e67-069d7acf6fdb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:28.465005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 965/1944] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:28 2022] POST /identity/v3/users => generated 285 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:29:28.472069 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8871c4f-45c4-470d-b542-f608f94de147 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:28.490658 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8871c4f-45c4-470d-b542-f608f94de147 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:28.499469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284be89-f9e1-4a8e-9260-3d1ace403d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:28.500658 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284be89-f9e1-4a8e-9260-3d1ace403d91 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:28.501735 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284be89-f9e1-4a8e-9260-3d1ace403d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:28.501997 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284be89-f9e1-4a8e-9260-3d1ace403d91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:28.507268 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b284be89-f9e1-4a8e-9260-3d1ace403d91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:28.507268 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b284be89-f9e1-4a8e-9260-3d1ace403d91 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:28.545001 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 980/1945] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:29:28.546114 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91aa67-6911-4d4f-8e67-069d7acf6fdb admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:28.564337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91aa67-6911-4d4f-8e67-069d7acf6fdb admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:28.575450 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70aa4e-38d2-40d5-a2e2-1cd3b4fc1307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:28.577825 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70aa4e-38d2-40d5-a2e2-1cd3b4fc1307 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:28.578303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70aa4e-38d2-40d5-a2e2-1cd3b4fc1307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:28.579039 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70aa4e-38d2-40d5-a2e2-1cd3b4fc1307 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:28.580842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70aa4e-38d2-40d5-a2e2-1cd3b4fc1307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:28.584259 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70aa4e-38d2-40d5-a2e2-1cd3b4fc1307 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:28.619842 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284be89-f9e1-4a8e-9260-3d1ace403d91 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:28.625655 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 966/1946] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:28.649471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284be89-f9e1-4a8e-9260-3d1ace403d91 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:28.676915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21b177-905d-43d6-9313-abe69f624996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:28.678594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21b177-905d-43d6-9313-abe69f624996 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:28.679088 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21b177-905d-43d6-9313-abe69f624996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:28.679583 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21b177-905d-43d6-9313-abe69f624996 admin admin] PATH_INFO: `/v3/projects/25099a55fa294d4dad5cc082ea5ba163/users/e8c219c8d81e44a1bbd7e1812881803b/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:28.724371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21b177-905d-43d6-9313-abe69f624996 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25099a55fa294d4dad5cc082ea5ba163, user_id=e8c219c8d81e44a1bbd7e1812881803b, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:28.730104 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21b177-905d-43d6-9313-abe69f624996 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:28.732110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70aa4e-38d2-40d5-a2e2-1cd3b4fc1307 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:28.738163 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1b70aa4e-38d2-40d5-a2e2-1cd3b4fc1307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:28.750718 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 981/1947] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:28 2022] PUT /identity/v3/projects/25099a55fa294d4dad5cc082ea5ba163/users/e8c219c8d81e44a1bbd7e1812881803b/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:28.770822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70aa4e-38d2-40d5-a2e2-1cd3b4fc1307 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:28.772098 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffebe3-04ce-4326-9b5f-7aa232d8ea1b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:28.772624 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffebe3-04ce-4326-9b5f-7aa232d8ea1b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:28.773116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffebe3-04ce-4326-9b5f-7aa232d8ea1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:28.843787 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b0ffebe3-04ce-4326-9b5f-7aa232d8ea1b None None] MFA Rules not processed for user `e8c219c8d81e44a1bbd7e1812881803b`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:28.847649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b0ffebe3-04ce-4326-9b5f-7aa232d8ea1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:28.898044 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2099543-008d-4d23-b90b-455e37949bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:28.899981 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2099543-008d-4d23-b90b-455e37949bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:28.900343 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2099543-008d-4d23-b90b-455e37949bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:28.901059 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2099543-008d-4d23-b90b-455e37949bba admin admin] PATH_INFO: `/v3/users/3440036e801f410fa6734744304db483` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:28.902944 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2099543-008d-4d23-b90b-455e37949bba admin admin] RBAC: Authorizing `identity:delete_user(user_id=3440036e801f410fa6734744304db483)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:28.907085 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2099543-008d-4d23-b90b-455e37949bba admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:29.006092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 982/1948] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:28 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:29.010378 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffebe3-04ce-4326-9b5f-7aa232d8ea1b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:29.013204 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:29.015306 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d2099543-008d-4d23-b90b-455e37949bba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3440036e801f410fa6734744304db483'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:29.016030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d2099543-008d-4d23-b90b-455e37949bba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3440036e801f410fa6734744304db483'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:29.027567 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffebe3-04ce-4326-9b5f-7aa232d8ea1b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:29.037264 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384974b-9b97-4d67-b863-ebbf05bd999a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:29.038294 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384974b-9b97-4d67-b863-ebbf05bd999a admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:29.038766 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384974b-9b97-4d67-b863-ebbf05bd999a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:29.039226 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384974b-9b97-4d67-b863-ebbf05bd999a admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:29.040115 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384974b-9b97-4d67-b863-ebbf05bd999a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:29.043356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384974b-9b97-4d67-b863-ebbf05bd999a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:29.044651 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d2099543-008d-4d23-b90b-455e37949bba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3440036e801f410fa6734744304db483'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:29.054812 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:29.056028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d2099543-008d-4d23-b90b-455e37949bba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3440036e801f410fa6734744304db483'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:29.057835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-d2099543-008d-4d23-b90b-455e37949bba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3440036e801f410fa6734744304db483'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:29.080773 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 967/1949] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:28 2022] DELETE /identity/v3/users/3440036e801f410fa6734744304db483 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:29.081916 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2099543-008d-4d23-b90b-455e37949bba admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:29.083400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d2099543-008d-4d23-b90b-455e37949bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:29.084313 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 983/1950] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:29 2022] POST /identity/v3/projects => generated 369 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:29:29.085850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727d156-b4f5-42bd-9e98-cd913974716a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:29.086276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727d156-b4f5-42bd-9e98-cd913974716a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:29.086416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727d156-b4f5-42bd-9e98-cd913974716a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:29.104883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-9727d156-b4f5-42bd-9e98-cd913974716a None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:29.106142 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9727d156-b4f5-42bd-9e98-cd913974716a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:29.160540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2099543-008d-4d23-b90b-455e37949bba admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:29.211369 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 984/1951] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:29.211369 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727d156-b4f5-42bd-9e98-cd913974716a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:29.219964 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9727d156-b4f5-42bd-9e98-cd913974716a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:29.257534 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727d156-b4f5-42bd-9e98-cd913974716a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:29.270552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d793f49-a676-4efd-9b21-5f1745715b6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:29.270836 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc49b3c-24ba-44ff-bc95-5659aea6cc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:29.271810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d793f49-a676-4efd-9b21-5f1745715b6b service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:29.271810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d793f49-a676-4efd-9b21-5f1745715b6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:29.272123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc49b3c-24ba-44ff-bc95-5659aea6cc39 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:29.272355 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d793f49-a676-4efd-9b21-5f1745715b6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:29.272455 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc49b3c-24ba-44ff-bc95-5659aea6cc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:29.272671 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc49b3c-24ba-44ff-bc95-5659aea6cc39 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:29.273439 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc49b3c-24ba-44ff-bc95-5659aea6cc39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:29.278525 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc49b3c-24ba-44ff-bc95-5659aea6cc39 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:29.293737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3d793f49-a676-4efd-9b21-5f1745715b6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:29.348017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 985/1952] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:29 2022] POST /identity/v3/users => generated 291 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:29.353196 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc49b3c-24ba-44ff-bc95-5659aea6cc39 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:29.359532 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ddc49b3c-24ba-44ff-bc95-5659aea6cc39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:29.443164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d793f49-a676-4efd-9b21-5f1745715b6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:29.453921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d793f49-a676-4efd-9b21-5f1745715b6b service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:29.468578 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 968/1953] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:29 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:29.469475 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d793f49-a676-4efd-9b21-5f1745715b6b service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:29.490777 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d793f49-a676-4efd-9b21-5f1745715b6b service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:29.506342 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c66a9-261a-45d7-bb33-91c8b65886f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:29.507020 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c66a9-261a-45d7-bb33-91c8b65886f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:29.507414 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c66a9-261a-45d7-bb33-91c8b65886f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:29.507706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c66a9-261a-45d7-bb33-91c8b65886f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:29.508444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c66a9-261a-45d7-bb33-91c8b65886f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:29.510359 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c66a9-261a-45d7-bb33-91c8b65886f5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:29.535300 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 969/1954] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:29.536233 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c66a9-261a-45d7-bb33-91c8b65886f5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:29.549287 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c66a9-261a-45d7-bb33-91c8b65886f5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:29.565238 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff884cb6-f889-46be-bf30-128d5fd160da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:29.567003 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff884cb6-f889-46be-bf30-128d5fd160da admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:29.567003 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff884cb6-f889-46be-bf30-128d5fd160da admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:29.567570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff884cb6-f889-46be-bf30-128d5fd160da admin admin] PATH_INFO: `/v3/projects/918a9256d3bf47b592d43624b1d96fad/users/2b320255669a48f897525c0bd7df29a6/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:29.595300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc49b3c-24ba-44ff-bc95-5659aea6cc39 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:29.598718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff884cb6-f889-46be-bf30-128d5fd160da admin admin] RBAC: Authorizing `identity:create_grant(project_id=918a9256d3bf47b592d43624b1d96fad, user_id=2b320255669a48f897525c0bd7df29a6, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:29.605934 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd82ff6-322b-4e3f-b6a4-32347ab3cd65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:29.606103 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff884cb6-f889-46be-bf30-128d5fd160da admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:29.606199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd82ff6-322b-4e3f-b6a4-32347ab3cd65 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:29.606199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd82ff6-322b-4e3f-b6a4-32347ab3cd65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:29.606199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd82ff6-322b-4e3f-b6a4-32347ab3cd65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:29.608323 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:29.620111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd82ff6-322b-4e3f-b6a4-32347ab3cd65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:29.621806 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd82ff6-322b-4e3f-b6a4-32347ab3cd65 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:29.625711 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 970/1955] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:29 2022] PUT /identity/v3/projects/918a9256d3bf47b592d43624b1d96fad/users/2b320255669a48f897525c0bd7df29a6/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:29.629467 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff884cb6-f889-46be-bf30-128d5fd160da admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:29.634602 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 986/1956] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:29.740088 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff884cb6-f889-46be-bf30-128d5fd160da admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:29.746123 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac70ca-6101-4849-90b0-d5a172fd0f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:29.747205 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac70ca-6101-4849-90b0-d5a172fd0f29 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:29.747585 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac70ca-6101-4849-90b0-d5a172fd0f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:29.747953 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac70ca-6101-4849-90b0-d5a172fd0f29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:29.749446 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac70ca-6101-4849-90b0-d5a172fd0f29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:29.752888 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac70ca-6101-4849-90b0-d5a172fd0f29 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:29.774270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 971/1957] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:29:29.774846 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd82ff6-322b-4e3f-b6a4-32347ab3cd65 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:29.800100 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd82ff6-322b-4e3f-b6a4-32347ab3cd65 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:29.828761 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd58c7-5a7b-4471-92ba-f6c2a89202da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:29.829793 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd58c7-5a7b-4471-92ba-f6c2a89202da admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:29.830374 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd58c7-5a7b-4471-92ba-f6c2a89202da admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:29.830852 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd58c7-5a7b-4471-92ba-f6c2a89202da admin admin] PATH_INFO: `/v3/projects/918a9256d3bf47b592d43624b1d96fad/users/2b320255669a48f897525c0bd7df29a6/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:29.835893 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd58c7-5a7b-4471-92ba-f6c2a89202da admin admin] RBAC: Authorizing `identity:create_grant(project_id=918a9256d3bf47b592d43624b1d96fad, user_id=2b320255669a48f897525c0bd7df29a6, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:29.844082 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd58c7-5a7b-4471-92ba-f6c2a89202da admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:29.863760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 987/1958] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:29 2022] PUT /identity/v3/projects/918a9256d3bf47b592d43624b1d96fad/users/2b320255669a48f897525c0bd7df29a6/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:29.875075 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab93f93-0399-4104-bdfe-e1d293ed0696 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:29.875425 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab93f93-0399-4104-bdfe-e1d293ed0696 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:29.875780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab93f93-0399-4104-bdfe-e1d293ed0696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:29.906357 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-bab93f93-0399-4104-bdfe-e1d293ed0696 None None] MFA Rules not processed for user `2b320255669a48f897525c0bd7df29a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:29.907213 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bab93f93-0399-4104-bdfe-e1d293ed0696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:29.999883 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 972/1959] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:29 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:30.020837 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd58c7-5a7b-4471-92ba-f6c2a89202da admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:30.023494 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a7dd58c7-5a7b-4471-92ba-f6c2a89202da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:30.130219 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd58c7-5a7b-4471-92ba-f6c2a89202da admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:30.134536 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b0b5a-f0a5-4ebc-9874-17c8f6e6eb56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:30.135214 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b0b5a-f0a5-4ebc-9874-17c8f6e6eb56 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:30.135496 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b0b5a-f0a5-4ebc-9874-17c8f6e6eb56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:30.135809 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b0b5a-f0a5-4ebc-9874-17c8f6e6eb56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:30.147150 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b0b5a-f0a5-4ebc-9874-17c8f6e6eb56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:30.148629 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b0b5a-f0a5-4ebc-9874-17c8f6e6eb56 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:30.152901 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab93f93-0399-4104-bdfe-e1d293ed0696 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:30.154914 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bab93f93-0399-4104-bdfe-e1d293ed0696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:30.159868 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 988/1960] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:30 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:29:30.209057 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b0b5a-f0a5-4ebc-9874-17c8f6e6eb56 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:30.214313 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:30.219173 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b0b5a-f0a5-4ebc-9874-17c8f6e6eb56 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:30.225068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353defab-dde4-4a7a-9914-c7f5db95f23a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:30.225758 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353defab-dde4-4a7a-9914-c7f5db95f23a service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:30.226058 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353defab-dde4-4a7a-9914-c7f5db95f23a service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:30.226346 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353defab-dde4-4a7a-9914-c7f5db95f23a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:30.231706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-353defab-dde4-4a7a-9914-c7f5db95f23a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:30.239330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab93f93-0399-4104-bdfe-e1d293ed0696 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:30.243879 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf437e1-81d2-4eb0-a823-5778d8e4c85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:30.244730 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf437e1-81d2-4eb0-a823-5778d8e4c85f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:30.245010 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf437e1-81d2-4eb0-a823-5778d8e4c85f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:30.245279 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf437e1-81d2-4eb0-a823-5778d8e4c85f admin admin] PATH_INFO: `/v3/projects/a281a7fa5a6947ab9b181d7e190ac68a` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:30.246476 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf437e1-81d2-4eb0-a823-5778d8e4c85f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a281a7fa5a6947ab9b181d7e190ac68a)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:30.249056 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf437e1-81d2-4eb0-a823-5778d8e4c85f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:30.265756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:30.294237 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353defab-dde4-4a7a-9914-c7f5db95f23a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:30.295856 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353defab-dde4-4a7a-9914-c7f5db95f23a service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:30.298941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fbf437e1-81d2-4eb0-a823-5778d8e4c85f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a281a7fa5a6947ab9b181d7e190ac68a', 'request_id': 'req-fbf437e1-81d2-4eb0-a823-5778d8e4c85f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:30.305162 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 989/1961] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:30 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:29:30.305300 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fbf437e1-81d2-4eb0-a823-5778d8e4c85f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a281a7fa5a6947ab9b181d7e190ac68a', 'request_id': 'req-fbf437e1-81d2-4eb0-a823-5778d8e4c85f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:30.311570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fbf437e1-81d2-4eb0-a823-5778d8e4c85f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a281a7fa5a6947ab9b181d7e190ac68a', 'request_id': 'req-fbf437e1-81d2-4eb0-a823-5778d8e4c85f'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:30.325195 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fbf437e1-81d2-4eb0-a823-5778d8e4c85f admin admin] The token cache is being invalidate because project a281a7fa5a6947ab9b181d7e190ac68a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:30.325548 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fbf437e1-81d2-4eb0-a823-5778d8e4c85f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:30.326259 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fbf437e1-81d2-4eb0-a823-5778d8e4c85f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:30.327802 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 973/1962] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:30 2022] DELETE /identity/v3/projects/a281a7fa5a6947ab9b181d7e190ac68a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:30.332018 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353defab-dde4-4a7a-9914-c7f5db95f23a service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:30.345698 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353defab-dde4-4a7a-9914-c7f5db95f23a service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:30.412002 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c9f34-1595-47a6-933a-df88854c42f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:30.412996 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c9f34-1595-47a6-933a-df88854c42f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:30.413460 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c9f34-1595-47a6-933a-df88854c42f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:30.413849 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c9f34-1595-47a6-933a-df88854c42f3 admin admin] PATH_INFO: `/v3/users/f6dab8c66d6241e68ac7c88b077afbbb` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:30.415153 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c9f34-1595-47a6-933a-df88854c42f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6dab8c66d6241e68ac7c88b077afbbb)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:30.417171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c9f34-1595-47a6-933a-df88854c42f3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:30.471976 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-104c9f34-1595-47a6-933a-df88854c42f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6dab8c66d6241e68ac7c88b077afbbb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:30.490959 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-104c9f34-1595-47a6-933a-df88854c42f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6dab8c66d6241e68ac7c88b077afbbb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:30.492237 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-104c9f34-1595-47a6-933a-df88854c42f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6dab8c66d6241e68ac7c88b077afbbb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:30.503335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-104c9f34-1595-47a6-933a-df88854c42f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6dab8c66d6241e68ac7c88b077afbbb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:30.504277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-104c9f34-1595-47a6-933a-df88854c42f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6dab8c66d6241e68ac7c88b077afbbb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:30.524999 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 990/1963] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:30 2022] DELETE /identity/v3/users/f6dab8c66d6241e68ac7c88b077afbbb => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:31.130927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf437e1-81d2-4eb0-a823-5778d8e4c85f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:31.136880 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fbf437e1-81d2-4eb0-a823-5778d8e4c85f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:31.158857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf437e1-81d2-4eb0-a823-5778d8e4c85f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:31.291496 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b551d3-c23e-47b6-aec5-b7bebb6e78b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:31.292388 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b551d3-c23e-47b6-aec5-b7bebb6e78b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:31.293655 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b551d3-c23e-47b6-aec5-b7bebb6e78b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:31.293949 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b551d3-c23e-47b6-aec5-b7bebb6e78b0 admin admin] PATH_INFO: `/v3/projects/56070ec453e84f49bf315c969f346f87` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:31.295669 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b551d3-c23e-47b6-aec5-b7bebb6e78b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56070ec453e84f49bf315c969f346f87)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:31.297773 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b551d3-c23e-47b6-aec5-b7bebb6e78b0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:31.355692 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a7b551d3-c23e-47b6-aec5-b7bebb6e78b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56070ec453e84f49bf315c969f346f87', 'request_id': 'req-a7b551d3-c23e-47b6-aec5-b7bebb6e78b0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:31.363438 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a7b551d3-c23e-47b6-aec5-b7bebb6e78b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56070ec453e84f49bf315c969f346f87', 'request_id': 'req-a7b551d3-c23e-47b6-aec5-b7bebb6e78b0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:31.369768 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a7b551d3-c23e-47b6-aec5-b7bebb6e78b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56070ec453e84f49bf315c969f346f87', 'request_id': 'req-a7b551d3-c23e-47b6-aec5-b7bebb6e78b0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:31.384223 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a7b551d3-c23e-47b6-aec5-b7bebb6e78b0 admin admin] The token cache is being invalidate because project 56070ec453e84f49bf315c969f346f87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:31.384446 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a7b551d3-c23e-47b6-aec5-b7bebb6e78b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:31.385090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a7b551d3-c23e-47b6-aec5-b7bebb6e78b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:31.386700 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 974/1964] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:31 2022] DELETE /identity/v3/projects/56070ec453e84f49bf315c969f346f87 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:31.392471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f5c24-0f45-4780-9308-93f7fa71a464 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:31.392967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f5c24-0f45-4780-9308-93f7fa71a464 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:31.393059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f5c24-0f45-4780-9308-93f7fa71a464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:31.411548 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c06f5c24-0f45-4780-9308-93f7fa71a464 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:31.412776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c06f5c24-0f45-4780-9308-93f7fa71a464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:31.429704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:31.475878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 991/1965] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:31.484365 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733bfc83-4316-43d4-8300-df20d8b78b9b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:31.484653 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733bfc83-4316-43d4-8300-df20d8b78b9b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:31.484932 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733bfc83-4316-43d4-8300-df20d8b78b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:31.504361 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-733bfc83-4316-43d4-8300-df20d8b78b9b None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:31.505637 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-733bfc83-4316-43d4-8300-df20d8b78b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:31.514133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 975/1966] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:29:31.518427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f5c24-0f45-4780-9308-93f7fa71a464 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:31.528491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f5c24-0f45-4780-9308-93f7fa71a464 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:31.533680 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851c0ad-6d63-4b1f-a1f2-6be633ac95a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:31.534332 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851c0ad-6d63-4b1f-a1f2-6be633ac95a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:31.534730 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851c0ad-6d63-4b1f-a1f2-6be633ac95a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:31.535017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851c0ad-6d63-4b1f-a1f2-6be633ac95a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:31.535746 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851c0ad-6d63-4b1f-a1f2-6be633ac95a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:29:31.536063 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851c0ad-6d63-4b1f-a1f2-6be633ac95a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:31.537697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851c0ad-6d63-4b1f-a1f2-6be633ac95a8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:31.550245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 992/1967] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:29:31.554540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733bfc83-4316-43d4-8300-df20d8b78b9b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:31.565186 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733bfc83-4316-43d4-8300-df20d8b78b9b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:31.570082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca1068-08cb-4b89-982a-7d24b5b0522c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:31.570759 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca1068-08cb-4b89-982a-7d24b5b0522c admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:31.571053 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca1068-08cb-4b89-982a-7d24b5b0522c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:31.571807 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca1068-08cb-4b89-982a-7d24b5b0522c admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:31.572682 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca1068-08cb-4b89-982a-7d24b5b0522c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:31.574655 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca1068-08cb-4b89-982a-7d24b5b0522c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:31.603196 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 976/1968] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:31 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:31.607115 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851c0ad-6d63-4b1f-a1f2-6be633ac95a8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:31.617653 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851c0ad-6d63-4b1f-a1f2-6be633ac95a8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:31.673201 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b623f6-fee3-4f7d-a8f7-3c7c764b7009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:31.673818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b623f6-fee3-4f7d-a8f7-3c7c764b7009 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:31.674106 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b623f6-fee3-4f7d-a8f7-3c7c764b7009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:31.674402 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b623f6-fee3-4f7d-a8f7-3c7c764b7009 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:31.675067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b623f6-fee3-4f7d-a8f7-3c7c764b7009 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:31.677000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b623f6-fee3-4f7d-a8f7-3c7c764b7009 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:31.696608 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 993/1969] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:31 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:31.700674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca1068-08cb-4b89-982a-7d24b5b0522c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:31.711138 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca1068-08cb-4b89-982a-7d24b5b0522c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:31.716459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c4224-c318-4ea4-9edf-a468ee861457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:31.717807 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c4224-c318-4ea4-9edf-a468ee861457 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:31.718090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c4224-c318-4ea4-9edf-a468ee861457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:31.718364 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c4224-c318-4ea4-9edf-a468ee861457 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:31.719089 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831c4224-c318-4ea4-9edf-a468ee861457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:31.720698 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831c4224-c318-4ea4-9edf-a468ee861457 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:31.722076 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:31.735245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 977/1970] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:31.738915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b623f6-fee3-4f7d-a8f7-3c7c764b7009 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:31.749126 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b623f6-fee3-4f7d-a8f7-3c7c764b7009 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:31.753715 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef4363-2805-407c-9b32-5e82e3453aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:31.754366 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef4363-2805-407c-9b32-5e82e3453aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:31.754656 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef4363-2805-407c-9b32-5e82e3453aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:31.754934 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef4363-2805-407c-9b32-5e82e3453aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:31.755655 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef4363-2805-407c-9b32-5e82e3453aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:31.757251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef4363-2805-407c-9b32-5e82e3453aa6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:31.767197 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 994/1971] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:31.771167 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c4224-c318-4ea4-9edf-a468ee861457 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:31.781346 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c4224-c318-4ea4-9edf-a468ee861457 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:31.786276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acffac4a-9726-47bc-a1d3-3304b1057341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:31.787150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acffac4a-9726-47bc-a1d3-3304b1057341 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:31.787437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acffac4a-9726-47bc-a1d3-3304b1057341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:31.787739 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acffac4a-9726-47bc-a1d3-3304b1057341 admin admin] PATH_INFO: `/v3/projects/49928f5f7e5b4d01b89c26304fe2e890/users/9a57932244664c249ea0704472458f70/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:31.802779 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acffac4a-9726-47bc-a1d3-3304b1057341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49928f5f7e5b4d01b89c26304fe2e890, user_id=9a57932244664c249ea0704472458f70, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:31.805436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acffac4a-9726-47bc-a1d3-3304b1057341 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:31.818236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 978/1972] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:31 2022] PUT /identity/v3/projects/49928f5f7e5b4d01b89c26304fe2e890/users/9a57932244664c249ea0704472458f70/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:31.828710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5f571-a9e9-444c-be3a-5261390eb0d3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:31.829077 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5f571-a9e9-444c-be3a-5261390eb0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:31.829286 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5f571-a9e9-444c-be3a-5261390eb0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:31.860225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c8d5f571-a9e9-444c-be3a-5261390eb0d3 None None] MFA Rules not processed for user `9a57932244664c249ea0704472458f70`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:31.861475 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c8d5f571-a9e9-444c-be3a-5261390eb0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:31.959658 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 995/1973] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:31.973738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acffac4a-9726-47bc-a1d3-3304b1057341 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:31.976297 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-acffac4a-9726-47bc-a1d3-3304b1057341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:32.040123 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acffac4a-9726-47bc-a1d3-3304b1057341 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:32.044091 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff4d046-fd85-4175-8963-e4cd4217deb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:32.044741 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4d046-fd85-4175-8963-e4cd4217deb0 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:32.045717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4d046-fd85-4175-8963-e4cd4217deb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:32.045979 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4d046-fd85-4175-8963-e4cd4217deb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:32.054822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff4d046-fd85-4175-8963-e4cd4217deb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:32.056263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff4d046-fd85-4175-8963-e4cd4217deb0 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:32.066805 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 979/1974] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:32.095691 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5f571-a9e9-444c-be3a-5261390eb0d3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:32.117431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:32.210805 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5f571-a9e9-444c-be3a-5261390eb0d3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:32.223661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca90348-22c2-4fda-8978-ed8b1b70144a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:32.224550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca90348-22c2-4fda-8978-ed8b1b70144a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:32.224839 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca90348-22c2-4fda-8978-ed8b1b70144a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:32.225221 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca90348-22c2-4fda-8978-ed8b1b70144a admin admin] PATH_INFO: `/v3/users/9a57932244664c249ea0704472458f70` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:32.237263 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca90348-22c2-4fda-8978-ed8b1b70144a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a57932244664c249ea0704472458f70)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:32.239440 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca90348-22c2-4fda-8978-ed8b1b70144a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:32.314042 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0ca90348-22c2-4fda-8978-ed8b1b70144a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a57932244664c249ea0704472458f70'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:32.332951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0ca90348-22c2-4fda-8978-ed8b1b70144a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a57932244664c249ea0704472458f70'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:32.334034 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0ca90348-22c2-4fda-8978-ed8b1b70144a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a57932244664c249ea0704472458f70'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:32.344335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0ca90348-22c2-4fda-8978-ed8b1b70144a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a57932244664c249ea0704472458f70'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:32.346012 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0ca90348-22c2-4fda-8978-ed8b1b70144a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a57932244664c249ea0704472458f70'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:32.363331 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 996/1975] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:32 2022] DELETE /identity/v3/users/9a57932244664c249ea0704472458f70 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:32.377439 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad81abf-f235-4f4a-bf3c-c742b51c8a5d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:32.377965 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad81abf-f235-4f4a-bf3c-c742b51c8a5d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:32.378306 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad81abf-f235-4f4a-bf3c-c742b51c8a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:32.399186 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-bad81abf-f235-4f4a-bf3c-c742b51c8a5d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:32.400529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bad81abf-f235-4f4a-bf3c-c742b51c8a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:32.536605 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 980/1976] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:32.545572 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca90348-22c2-4fda-8978-ed8b1b70144a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:32.547131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0ca90348-22c2-4fda-8978-ed8b1b70144a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:32.673368 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca90348-22c2-4fda-8978-ed8b1b70144a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:32.681553 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6418a-23c1-47ce-993a-12e377b1a194 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:32.682704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6418a-23c1-47ce-993a-12e377b1a194 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:32.683256 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6418a-23c1-47ce-993a-12e377b1a194 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:32.683595 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6418a-23c1-47ce-993a-12e377b1a194 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:32.698890 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6418a-23c1-47ce-993a-12e377b1a194 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:32.700918 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6418a-23c1-47ce-993a-12e377b1a194 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:32.719578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 997/1977] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:33.090539 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad81abf-f235-4f4a-bf3c-c742b51c8a5d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:33.091968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bad81abf-f235-4f4a-bf3c-c742b51c8a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:33.105321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad81abf-f235-4f4a-bf3c-c742b51c8a5d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:33.111653 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c973f-b01c-43f5-a543-526067f1da77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:33.112593 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c973f-b01c-43f5-a543-526067f1da77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:33.112873 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c973f-b01c-43f5-a543-526067f1da77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:33.113202 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c973f-b01c-43f5-a543-526067f1da77 admin admin] PATH_INFO: `/v3/projects/49928f5f7e5b4d01b89c26304fe2e890` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:33.114500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c973f-b01c-43f5-a543-526067f1da77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49928f5f7e5b4d01b89c26304fe2e890)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:33.116436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c973f-b01c-43f5-a543-526067f1da77 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:33.171456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e18c973f-b01c-43f5-a543-526067f1da77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49928f5f7e5b4d01b89c26304fe2e890', 'request_id': 'req-e18c973f-b01c-43f5-a543-526067f1da77'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:33.178257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e18c973f-b01c-43f5-a543-526067f1da77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49928f5f7e5b4d01b89c26304fe2e890', 'request_id': 'req-e18c973f-b01c-43f5-a543-526067f1da77'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:33.185288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e18c973f-b01c-43f5-a543-526067f1da77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49928f5f7e5b4d01b89c26304fe2e890', 'request_id': 'req-e18c973f-b01c-43f5-a543-526067f1da77'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:33.198379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:33.212064 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e18c973f-b01c-43f5-a543-526067f1da77 admin admin] The token cache is being invalidate because project 49928f5f7e5b4d01b89c26304fe2e890 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:33.212394 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e18c973f-b01c-43f5-a543-526067f1da77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:33.214102 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e18c973f-b01c-43f5-a543-526067f1da77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:33.219787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 981/1978] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:33 2022] DELETE /identity/v3/projects/49928f5f7e5b4d01b89c26304fe2e890 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:33.225004 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e134644-2754-4b87-b3c4-0765ea87b9ce None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:33.225804 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e134644-2754-4b87-b3c4-0765ea87b9ce None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:33.226295 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e134644-2754-4b87-b3c4-0765ea87b9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:33.249987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3e134644-2754-4b87-b3c4-0765ea87b9ce None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:33.251260 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3e134644-2754-4b87-b3c4-0765ea87b9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:33.293359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:33.329443 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 998/1979] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:29:33.338298 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd103529-97f9-47aa-a89a-0f9f28fd7dc1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:33.338645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd103529-97f9-47aa-a89a-0f9f28fd7dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:33.338934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd103529-97f9-47aa-a89a-0f9f28fd7dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:33.359075 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fd103529-97f9-47aa-a89a-0f9f28fd7dc1 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:33.360306 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fd103529-97f9-47aa-a89a-0f9f28fd7dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:33.368761 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 982/1980] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:33.372902 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e134644-2754-4b87-b3c4-0765ea87b9ce None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:33.383040 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e134644-2754-4b87-b3c4-0765ea87b9ce None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:33.388367 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0b422-95ae-4ab5-98c9-85d263bad191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:33.389322 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0b422-95ae-4ab5-98c9-85d263bad191 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:33.389322 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0b422-95ae-4ab5-98c9-85d263bad191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:33.389625 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0b422-95ae-4ab5-98c9-85d263bad191 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:33.390343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c0b422-95ae-4ab5-98c9-85d263bad191 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:29:33.390656 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c0b422-95ae-4ab5-98c9-85d263bad191 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:33.392280 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c0b422-95ae-4ab5-98c9-85d263bad191 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:33.405560 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 999/1981] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:29:33.409828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd103529-97f9-47aa-a89a-0f9f28fd7dc1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:33.419007 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd103529-97f9-47aa-a89a-0f9f28fd7dc1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:33.425010 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d6f21-0806-4cf3-94a9-e47e81018a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:33.425689 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d6f21-0806-4cf3-94a9-e47e81018a03 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:33.425973 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d6f21-0806-4cf3-94a9-e47e81018a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:33.426250 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d6f21-0806-4cf3-94a9-e47e81018a03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:33.426978 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d6f21-0806-4cf3-94a9-e47e81018a03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:33.428883 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d6f21-0806-4cf3-94a9-e47e81018a03 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:33.467940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 983/1982] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:33 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:33.472201 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0b422-95ae-4ab5-98c9-85d263bad191 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:33.483298 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0b422-95ae-4ab5-98c9-85d263bad191 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:33.553907 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762de66-8ca5-4bf4-bce3-a054b8b615c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:33.554600 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762de66-8ca5-4bf4-bce3-a054b8b615c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:33.554913 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762de66-8ca5-4bf4-bce3-a054b8b615c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:33.555206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762de66-8ca5-4bf4-bce3-a054b8b615c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:33.556044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8762de66-8ca5-4bf4-bce3-a054b8b615c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:33.558242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8762de66-8ca5-4bf4-bce3-a054b8b615c4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:33.583830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1000/1983] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:33 2022] POST /identity/v3/users => generated 291 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:33.599455 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d6f21-0806-4cf3-94a9-e47e81018a03 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:33.603871 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d6f21-0806-4cf3-94a9-e47e81018a03 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:33.621410 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257a57d-fbb9-4e17-8183-a1feff11a24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:33.622605 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257a57d-fbb9-4e17-8183-a1feff11a24c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:33.623180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257a57d-fbb9-4e17-8183-a1feff11a24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:33.624343 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257a57d-fbb9-4e17-8183-a1feff11a24c admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:33.625538 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257a57d-fbb9-4e17-8183-a1feff11a24c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:33.627563 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257a57d-fbb9-4e17-8183-a1feff11a24c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:33.659854 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 984/1984] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:33.664729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762de66-8ca5-4bf4-bce3-a054b8b615c4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:33.679486 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762de66-8ca5-4bf4-bce3-a054b8b615c4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:33.688433 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb322b45-1299-4fe2-81eb-f8761e39bba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:33.689336 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb322b45-1299-4fe2-81eb-f8761e39bba0 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:33.689812 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb322b45-1299-4fe2-81eb-f8761e39bba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:33.690407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb322b45-1299-4fe2-81eb-f8761e39bba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:33.693007 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb322b45-1299-4fe2-81eb-f8761e39bba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:33.695291 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb322b45-1299-4fe2-81eb-f8761e39bba0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:33.710767 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1001/1985] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:33.714958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257a57d-fbb9-4e17-8183-a1feff11a24c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:33.729669 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257a57d-fbb9-4e17-8183-a1feff11a24c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:33.736618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af98fe-4599-42e2-998c-53ffb634427a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:33.737730 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af98fe-4599-42e2-998c-53ffb634427a admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:33.738847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af98fe-4599-42e2-998c-53ffb634427a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:33.738847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af98fe-4599-42e2-998c-53ffb634427a admin admin] PATH_INFO: `/v3/projects/175743f30d314455a3bead1c50750fbe/users/4128f8b7190e455b8dc599a404e3608c/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:33.765695 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af98fe-4599-42e2-998c-53ffb634427a admin admin] RBAC: Authorizing `identity:create_grant(project_id=175743f30d314455a3bead1c50750fbe, user_id=4128f8b7190e455b8dc599a404e3608c, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:33.768672 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af98fe-4599-42e2-998c-53ffb634427a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:33.798165 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 985/1986] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:33 2022] PUT /identity/v3/projects/175743f30d314455a3bead1c50750fbe/users/4128f8b7190e455b8dc599a404e3608c/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:33.802816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655073f-7e71-4432-b03f-508365cea388 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:33.803399 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655073f-7e71-4432-b03f-508365cea388 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:33.803787 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655073f-7e71-4432-b03f-508365cea388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:33.840646 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d655073f-7e71-4432-b03f-508365cea388 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:33.842245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d655073f-7e71-4432-b03f-508365cea388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:33.960664 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1002/1987] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:33.971733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af98fe-4599-42e2-998c-53ffb634427a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:33.973865 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f7af98fe-4599-42e2-998c-53ffb634427a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:34.081317 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af98fe-4599-42e2-998c-53ffb634427a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:34.087711 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c0f40-3761-47fa-8070-569421c55ed2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:34.088794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c0f40-3761-47fa-8070-569421c55ed2 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:34.089271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c0f40-3761-47fa-8070-569421c55ed2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:34.089825 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c0f40-3761-47fa-8070-569421c55ed2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:34.104267 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c0f40-3761-47fa-8070-569421c55ed2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:34.105958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c0f40-3761-47fa-8070-569421c55ed2 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:34.128057 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 986/1988] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:34.783274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655073f-7e71-4432-b03f-508365cea388 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:34.785530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d655073f-7e71-4432-b03f-508365cea388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:34.797788 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655073f-7e71-4432-b03f-508365cea388 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:34.802343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b014ef9-98c3-4252-9d7a-752083cf6985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:34.803402 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b014ef9-98c3-4252-9d7a-752083cf6985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:34.803693 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b014ef9-98c3-4252-9d7a-752083cf6985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:34.803978 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b014ef9-98c3-4252-9d7a-752083cf6985 admin admin] PATH_INFO: `/v3/users/e8c219c8d81e44a1bbd7e1812881803b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:34.805202 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b014ef9-98c3-4252-9d7a-752083cf6985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8c219c8d81e44a1bbd7e1812881803b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:34.807341 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b014ef9-98c3-4252-9d7a-752083cf6985 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:34.860055 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:34.873287 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3b014ef9-98c3-4252-9d7a-752083cf6985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8c219c8d81e44a1bbd7e1812881803b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:34.891510 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3b014ef9-98c3-4252-9d7a-752083cf6985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8c219c8d81e44a1bbd7e1812881803b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:34.892471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3b014ef9-98c3-4252-9d7a-752083cf6985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8c219c8d81e44a1bbd7e1812881803b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:34.902934 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3b014ef9-98c3-4252-9d7a-752083cf6985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8c219c8d81e44a1bbd7e1812881803b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:34.905271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3b014ef9-98c3-4252-9d7a-752083cf6985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8c219c8d81e44a1bbd7e1812881803b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:34.929538 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1003/1989] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:34 2022] DELETE /identity/v3/users/e8c219c8d81e44a1bbd7e1812881803b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:34.934170 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aefaa40-d799-49d4-869e-ceba5c8c6246 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:34.934962 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aefaa40-d799-49d4-869e-ceba5c8c6246 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:34.935713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aefaa40-d799-49d4-869e-ceba5c8c6246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:34.968610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1aefaa40-d799-49d4-869e-ceba5c8c6246 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:34.969946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1aefaa40-d799-49d4-869e-ceba5c8c6246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:35.045289 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878fe2c-baec-42b9-8987-4946b4e6c43a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:35.045743 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878fe2c-baec-42b9-8987-4946b4e6c43a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:35.046041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878fe2c-baec-42b9-8987-4946b4e6c43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:35.080329 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b878fe2c-baec-42b9-8987-4946b4e6c43a None None] MFA Rules not processed for user `4128f8b7190e455b8dc599a404e3608c`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:35.081568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b878fe2c-baec-42b9-8987-4946b4e6c43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:35.101631 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 987/1990] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:35.108575 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aefaa40-d799-49d4-869e-ceba5c8c6246 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:35.111129 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1aefaa40-d799-49d4-869e-ceba5c8c6246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:35.189368 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1004/1991] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:35.195085 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:35.219263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aefaa40-d799-49d4-869e-ceba5c8c6246 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:35.223660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5513291a-63b7-49ad-96ec-6ddb540a4e4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:35.224325 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513291a-63b7-49ad-96ec-6ddb540a4e4c service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:35.224611 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513291a-63b7-49ad-96ec-6ddb540a4e4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:35.224937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513291a-63b7-49ad-96ec-6ddb540a4e4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:35.230059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878fe2c-baec-42b9-8987-4946b4e6c43a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:35.231612 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b878fe2c-baec-42b9-8987-4946b4e6c43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:35.240630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5513291a-63b7-49ad-96ec-6ddb540a4e4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:35.242879 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5513291a-63b7-49ad-96ec-6ddb540a4e4c service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:35.259530 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 988/1992] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:35.344107 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878fe2c-baec-42b9-8987-4946b4e6c43a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:35.352008 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e918cb42-e7a4-4059-b516-5ab5a677d6d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:35.352692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918cb42-e7a4-4059-b516-5ab5a677d6d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:35.352987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918cb42-e7a4-4059-b516-5ab5a677d6d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:35.353274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918cb42-e7a4-4059-b516-5ab5a677d6d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:35.365627 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e918cb42-e7a4-4059-b516-5ab5a677d6d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:35.367309 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e918cb42-e7a4-4059-b516-5ab5a677d6d1 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:35.382267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1005/1993] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:29:35.427555 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5513291a-63b7-49ad-96ec-6ddb540a4e4c service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:35.429375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5513291a-63b7-49ad-96ec-6ddb540a4e4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:35.446056 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5513291a-63b7-49ad-96ec-6ddb540a4e4c service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:35.451205 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964399cf-b6f0-4838-9ca4-5f34334069e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:35.452792 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964399cf-b6f0-4838-9ca4-5f34334069e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:35.453102 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964399cf-b6f0-4838-9ca4-5f34334069e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:35.453410 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964399cf-b6f0-4838-9ca4-5f34334069e9 admin admin] PATH_INFO: `/v3/users/f5330763868b42a08e5016a56dcdd59b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:35.454689 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964399cf-b6f0-4838-9ca4-5f34334069e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5330763868b42a08e5016a56dcdd59b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:35.456636 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964399cf-b6f0-4838-9ca4-5f34334069e9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:35.564760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-964399cf-b6f0-4838-9ca4-5f34334069e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5330763868b42a08e5016a56dcdd59b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:35.567135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-964399cf-b6f0-4838-9ca4-5f34334069e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5330763868b42a08e5016a56dcdd59b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:35.608353 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-964399cf-b6f0-4838-9ca4-5f34334069e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5330763868b42a08e5016a56dcdd59b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:35.633182 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-964399cf-b6f0-4838-9ca4-5f34334069e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5330763868b42a08e5016a56dcdd59b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:35.633889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-964399cf-b6f0-4838-9ca4-5f34334069e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5330763868b42a08e5016a56dcdd59b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:35.652287 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 989/1994] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:35 2022] DELETE /identity/v3/users/f5330763868b42a08e5016a56dcdd59b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:35.657972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a30e3-e4a6-406f-a581-166110cdc4d3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:35.658269 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a30e3-e4a6-406f-a581-166110cdc4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:35.658558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a30e3-e4a6-406f-a581-166110cdc4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:35.684526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-010a30e3-e4a6-406f-a581-166110cdc4d3 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:35.687470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-010a30e3-e4a6-406f-a581-166110cdc4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:35.787971 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:35.820837 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1006/1995] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:35.836237 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964399cf-b6f0-4838-9ca4-5f34334069e9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:35.837842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-964399cf-b6f0-4838-9ca4-5f34334069e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:35.907295 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a30e3-e4a6-406f-a581-166110cdc4d3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:35.908903 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-010a30e3-e4a6-406f-a581-166110cdc4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:35.924165 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a30e3-e4a6-406f-a581-166110cdc4d3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:35.929208 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbae8ce-d6dd-4074-99d9-1736f607db3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:35.930456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbae8ce-d6dd-4074-99d9-1736f607db3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:35.930946 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbae8ce-d6dd-4074-99d9-1736f607db3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:35.931295 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbae8ce-d6dd-4074-99d9-1736f607db3d admin admin] PATH_INFO: `/v3/projects/25099a55fa294d4dad5cc082ea5ba163` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:35.932831 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbae8ce-d6dd-4074-99d9-1736f607db3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=25099a55fa294d4dad5cc082ea5ba163)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:35.935291 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbae8ce-d6dd-4074-99d9-1736f607db3d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:35.954325 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964399cf-b6f0-4838-9ca4-5f34334069e9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:35.958437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd36aa-abb5-42a9-a785-ea430016fae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:35.959441 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd36aa-abb5-42a9-a785-ea430016fae5 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:35.959729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd36aa-abb5-42a9-a785-ea430016fae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:35.960026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd36aa-abb5-42a9-a785-ea430016fae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:35.971524 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccd36aa-abb5-42a9-a785-ea430016fae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:35.972968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccd36aa-abb5-42a9-a785-ea430016fae5 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:35.983873 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 990/1996] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:29:35.990866 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3fbae8ce-d6dd-4074-99d9-1736f607db3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25099a55fa294d4dad5cc082ea5ba163', 'request_id': 'req-3fbae8ce-d6dd-4074-99d9-1736f607db3d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:35.997856 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3fbae8ce-d6dd-4074-99d9-1736f607db3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25099a55fa294d4dad5cc082ea5ba163', 'request_id': 'req-3fbae8ce-d6dd-4074-99d9-1736f607db3d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:36.006052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3fbae8ce-d6dd-4074-99d9-1736f607db3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25099a55fa294d4dad5cc082ea5ba163', 'request_id': 'req-3fbae8ce-d6dd-4074-99d9-1736f607db3d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:36.020871 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3fbae8ce-d6dd-4074-99d9-1736f607db3d admin admin] The token cache is being invalidate because project 25099a55fa294d4dad5cc082ea5ba163 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:36.021301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3fbae8ce-d6dd-4074-99d9-1736f607db3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:36.022318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-3fbae8ce-d6dd-4074-99d9-1736f607db3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:36.024483 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1007/1997] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:35 2022] DELETE /identity/v3/projects/25099a55fa294d4dad5cc082ea5ba163 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:36.028274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd36aa-abb5-42a9-a785-ea430016fae5 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:36.041291 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd36aa-abb5-42a9-a785-ea430016fae5 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:36.110602 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:36.119671 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce2935-0f37-4627-a323-5cbe2089cb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:36.121124 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce2935-0f37-4627-a323-5cbe2089cb6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:36.121658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce2935-0f37-4627-a323-5cbe2089cb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:36.122108 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce2935-0f37-4627-a323-5cbe2089cb6f admin admin] PATH_INFO: `/v3/users/2b320255669a48f897525c0bd7df29a6` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:36.124041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce2935-0f37-4627-a323-5cbe2089cb6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b320255669a48f897525c0bd7df29a6)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:36.126133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce2935-0f37-4627-a323-5cbe2089cb6f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:36.187647 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e5ce2935-0f37-4627-a323-5cbe2089cb6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b320255669a48f897525c0bd7df29a6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:36.188956 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e5ce2935-0f37-4627-a323-5cbe2089cb6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b320255669a48f897525c0bd7df29a6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:36.207465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e5ce2935-0f37-4627-a323-5cbe2089cb6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b320255669a48f897525c0bd7df29a6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:36.217257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e5ce2935-0f37-4627-a323-5cbe2089cb6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b320255669a48f897525c0bd7df29a6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:36.218082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e5ce2935-0f37-4627-a323-5cbe2089cb6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b320255669a48f897525c0bd7df29a6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:36.233190 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 991/1998] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:36 2022] DELETE /identity/v3/users/2b320255669a48f897525c0bd7df29a6 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:36.345990 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbae8ce-d6dd-4074-99d9-1736f607db3d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:36.345990 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3fbae8ce-d6dd-4074-99d9-1736f607db3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:36.363145 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbae8ce-d6dd-4074-99d9-1736f607db3d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:36.508352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd7804e-89b2-457e-9d0b-8d587f48f5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:36.510303 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7804e-89b2-457e-9d0b-8d587f48f5be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:36.510664 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7804e-89b2-457e-9d0b-8d587f48f5be admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:36.511017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7804e-89b2-457e-9d0b-8d587f48f5be admin admin] PATH_INFO: `/v3/projects/93dbe78cd01b4b35a0f2677f0524f540` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:36.512397 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd7804e-89b2-457e-9d0b-8d587f48f5be admin admin] RBAC: Authorizing `identity:delete_project(project_id=93dbe78cd01b4b35a0f2677f0524f540)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:36.514550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd7804e-89b2-457e-9d0b-8d587f48f5be admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:36.584302 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7dd7804e-89b2-457e-9d0b-8d587f48f5be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93dbe78cd01b4b35a0f2677f0524f540', 'request_id': 'req-7dd7804e-89b2-457e-9d0b-8d587f48f5be'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:36.592548 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7dd7804e-89b2-457e-9d0b-8d587f48f5be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93dbe78cd01b4b35a0f2677f0524f540', 'request_id': 'req-7dd7804e-89b2-457e-9d0b-8d587f48f5be'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:36.599744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7dd7804e-89b2-457e-9d0b-8d587f48f5be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93dbe78cd01b4b35a0f2677f0524f540', 'request_id': 'req-7dd7804e-89b2-457e-9d0b-8d587f48f5be'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:36.617092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7dd7804e-89b2-457e-9d0b-8d587f48f5be admin admin] The token cache is being invalidate because project 93dbe78cd01b4b35a0f2677f0524f540 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:36.617616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7dd7804e-89b2-457e-9d0b-8d587f48f5be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:36.618526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7dd7804e-89b2-457e-9d0b-8d587f48f5be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:36.621806 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1008/1999] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:36 2022] DELETE /identity/v3/projects/93dbe78cd01b4b35a0f2677f0524f540 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:36.625224 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce2935-0f37-4627-a323-5cbe2089cb6f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:36.628201 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e5ce2935-0f37-4627-a323-5cbe2089cb6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:36.642023 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce2935-0f37-4627-a323-5cbe2089cb6f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:36.664992 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd7804e-89b2-457e-9d0b-8d587f48f5be admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:36.666500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7dd7804e-89b2-457e-9d0b-8d587f48f5be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:36.674984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:36.774077 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd11d63-61d9-4f4c-a647-dd3f7dea1d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:36.774982 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd11d63-61d9-4f4c-a647-dd3f7dea1d18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:36.775272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd11d63-61d9-4f4c-a647-dd3f7dea1d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:36.775551 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd11d63-61d9-4f4c-a647-dd3f7dea1d18 admin admin] PATH_INFO: `/v3/users/8b85b9be871245f08ec3dbab4c9befe1` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:36.776855 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd11d63-61d9-4f4c-a647-dd3f7dea1d18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b85b9be871245f08ec3dbab4c9befe1)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:36.779015 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd11d63-61d9-4f4c-a647-dd3f7dea1d18 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:36.820283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd7804e-89b2-457e-9d0b-8d587f48f5be admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:36.837494 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81330be5-e3f3-4953-bda3-3d4a45c4f257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:36.839951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81330be5-e3f3-4953-bda3-3d4a45c4f257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:36.840486 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81330be5-e3f3-4953-bda3-3d4a45c4f257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:36.843003 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81330be5-e3f3-4953-bda3-3d4a45c4f257 admin admin] PATH_INFO: `/v3/projects/918a9256d3bf47b592d43624b1d96fad` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:36.847192 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81330be5-e3f3-4953-bda3-3d4a45c4f257 admin admin] RBAC: Authorizing `identity:delete_project(project_id=918a9256d3bf47b592d43624b1d96fad)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:36.849344 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81330be5-e3f3-4953-bda3-3d4a45c4f257 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:36.878633 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ffd11d63-61d9-4f4c-a647-dd3f7dea1d18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b85b9be871245f08ec3dbab4c9befe1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:36.880946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ffd11d63-61d9-4f4c-a647-dd3f7dea1d18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b85b9be871245f08ec3dbab4c9befe1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:36.899476 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ffd11d63-61d9-4f4c-a647-dd3f7dea1d18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b85b9be871245f08ec3dbab4c9befe1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:36.914439 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ffd11d63-61d9-4f4c-a647-dd3f7dea1d18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b85b9be871245f08ec3dbab4c9befe1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:36.915144 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ffd11d63-61d9-4f4c-a647-dd3f7dea1d18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b85b9be871245f08ec3dbab4c9befe1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:36.921636 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:36.939241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 992/2000] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:36 2022] DELETE /identity/v3/users/8b85b9be871245f08ec3dbab4c9befe1 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:36.939241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9dba3-81ce-437e-9b0c-eb09092c612f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:36.939241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9dba3-81ce-437e-9b0c-eb09092c612f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:36.939241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9dba3-81ce-437e-9b0c-eb09092c612f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:36.939773 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-81330be5-e3f3-4953-bda3-3d4a45c4f257 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '918a9256d3bf47b592d43624b1d96fad', 'request_id': 'req-81330be5-e3f3-4953-bda3-3d4a45c4f257'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:36.946849 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-81330be5-e3f3-4953-bda3-3d4a45c4f257 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '918a9256d3bf47b592d43624b1d96fad', 'request_id': 'req-81330be5-e3f3-4953-bda3-3d4a45c4f257'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:36.954572 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-81330be5-e3f3-4953-bda3-3d4a45c4f257 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '918a9256d3bf47b592d43624b1d96fad', 'request_id': 'req-81330be5-e3f3-4953-bda3-3d4a45c4f257'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:36.968735 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a4b9dba3-81ce-437e-9b0c-eb09092c612f None None] MFA Rules not processed for user `4128f8b7190e455b8dc599a404e3608c`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:36.969999 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a4b9dba3-81ce-437e-9b0c-eb09092c612f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:36.970556 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-81330be5-e3f3-4953-bda3-3d4a45c4f257 admin admin] The token cache is being invalidate because project 918a9256d3bf47b592d43624b1d96fad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:36.971185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-81330be5-e3f3-4953-bda3-3d4a45c4f257 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:36.971926 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-81330be5-e3f3-4953-bda3-3d4a45c4f257 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:36.987458 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1009/2001] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:36 2022] DELETE /identity/v3/projects/918a9256d3bf47b592d43624b1d96fad => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:36.992312 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedde5b-69d0-4ab2-a978-cd8c751b9219 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:36.992731 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedde5b-69d0-4ab2-a978-cd8c751b9219 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:36.993231 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedde5b-69d0-4ab2-a978-cd8c751b9219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:37.022384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ceedde5b-69d0-4ab2-a978-cd8c751b9219 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:37.024327 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ceedde5b-69d0-4ab2-a978-cd8c751b9219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:37.055189 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 993/2002] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:37.120545 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1010/2003] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:29:37.129187 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506da38-864b-41d2-b023-6a273ff8c371 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:37.129461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506da38-864b-41d2-b023-6a273ff8c371 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:37.130339 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506da38-864b-41d2-b023-6a273ff8c371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:37.159859 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c506da38-864b-41d2-b023-6a273ff8c371 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:37.161179 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c506da38-864b-41d2-b023-6a273ff8c371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:37.177621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 994/2004] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:37.179291 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceedde5b-69d0-4ab2-a978-cd8c751b9219 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:37.190713 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceedde5b-69d0-4ab2-a978-cd8c751b9219 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:37.195998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54c0d0-577d-405d-baee-b223f0571381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:37.196669 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54c0d0-577d-405d-baee-b223f0571381 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:37.196959 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54c0d0-577d-405d-baee-b223f0571381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:37.197619 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54c0d0-577d-405d-baee-b223f0571381 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:37.198335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d54c0d0-577d-405d-baee-b223f0571381 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:29:37.198645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d54c0d0-577d-405d-baee-b223f0571381 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:37.200263 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d54c0d0-577d-405d-baee-b223f0571381 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:37.214041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1011/2005] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:29:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:29:37.219299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506da38-864b-41d2-b023-6a273ff8c371 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:37.228660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506da38-864b-41d2-b023-6a273ff8c371 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:37.234218 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0defd7-07a9-45a0-ad1c-c21e4d5a01b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:37.234875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0defd7-07a9-45a0-ad1c-c21e4d5a01b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:37.235158 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0defd7-07a9-45a0-ad1c-c21e4d5a01b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:37.235430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0defd7-07a9-45a0-ad1c-c21e4d5a01b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:37.236116 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0defd7-07a9-45a0-ad1c-c21e4d5a01b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:37.238062 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0defd7-07a9-45a0-ad1c-c21e4d5a01b7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:37.268266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 995/2006] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:37 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:37.270315 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54c0d0-577d-405d-baee-b223f0571381 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:37.284918 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54c0d0-577d-405d-baee-b223f0571381 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:37.312599 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:37.391926 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6c35a-35bd-43d5-b98d-bc0a01687669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:37.394557 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6c35a-35bd-43d5-b98d-bc0a01687669 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:37.399808 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6c35a-35bd-43d5-b98d-bc0a01687669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:37.400138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6c35a-35bd-43d5-b98d-bc0a01687669 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:37.400870 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6c35a-35bd-43d5-b98d-bc0a01687669 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:37.402877 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6c35a-35bd-43d5-b98d-bc0a01687669 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:37.429723 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1012/2007] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:37 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:37.438595 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0defd7-07a9-45a0-ad1c-c21e4d5a01b7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:37.464006 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0defd7-07a9-45a0-ad1c-c21e4d5a01b7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:37.476760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e324d-be16-4bf4-a4ad-4a8a66ae9ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:37.478313 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e324d-be16-4bf4-a4ad-4a8a66ae9ebf admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:37.478782 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e324d-be16-4bf4-a4ad-4a8a66ae9ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:37.479595 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e324d-be16-4bf4-a4ad-4a8a66ae9ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:37.481782 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e324d-be16-4bf4-a4ad-4a8a66ae9ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:37.484868 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e324d-be16-4bf4-a4ad-4a8a66ae9ebf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:37.503171 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 996/2008] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:29:37.507605 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6c35a-35bd-43d5-b98d-bc0a01687669 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:37.529487 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6c35a-35bd-43d5-b98d-bc0a01687669 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:37.535743 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661761b-c9d5-453e-a226-b3324ee481f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:37.536825 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661761b-c9d5-453e-a226-b3324ee481f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:37.537594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661761b-c9d5-453e-a226-b3324ee481f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:37.538240 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661761b-c9d5-453e-a226-b3324ee481f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:37.539191 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1661761b-c9d5-453e-a226-b3324ee481f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:37.541207 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1661761b-c9d5-453e-a226-b3324ee481f2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:37.556196 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1013/2009] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:37.566630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e324d-be16-4bf4-a4ad-4a8a66ae9ebf admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:37.571243 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9d0e324d-be16-4bf4-a4ad-4a8a66ae9ebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:37.586810 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661761b-c9d5-453e-a226-b3324ee481f2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:37.601951 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e324d-be16-4bf4-a4ad-4a8a66ae9ebf admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:37.619438 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661761b-c9d5-453e-a226-b3324ee481f2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:37.630184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9993f-d9c3-497d-89aa-56c7a23cb2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:37.631267 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d446d-7695-4b29-b3fc-94dbae110a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:37.632298 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9993f-d9c3-497d-89aa-56c7a23cb2ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:37.632601 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9993f-d9c3-497d-89aa-56c7a23cb2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:37.633455 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d446d-7695-4b29-b3fc-94dbae110a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:37.633770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d446d-7695-4b29-b3fc-94dbae110a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:37.634050 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d446d-7695-4b29-b3fc-94dbae110a7d admin admin] PATH_INFO: `/v3/projects/387bb82a80fc434eb46a424c7c544762` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:37.635273 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9993f-d9c3-497d-89aa-56c7a23cb2ce admin admin] PATH_INFO: `/v3/projects/e775a32af8b64595bbd5cb1d2f1545f0/users/aa5417789b27472bb39d85a08253100b/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:37.636855 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d446d-7695-4b29-b3fc-94dbae110a7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=387bb82a80fc434eb46a424c7c544762)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:37.642304 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d446d-7695-4b29-b3fc-94dbae110a7d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:37.684830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea9993f-d9c3-497d-89aa-56c7a23cb2ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=e775a32af8b64595bbd5cb1d2f1545f0, user_id=aa5417789b27472bb39d85a08253100b, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:37.694292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea9993f-d9c3-497d-89aa-56c7a23cb2ce admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:37.729894 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1014/2010] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:37 2022] PUT /identity/v3/projects/e775a32af8b64595bbd5cb1d2f1545f0/users/aa5417789b27472bb39d85a08253100b/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:37.740697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61060b6-15da-43c7-a85c-e964e92085ee None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:37.740992 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61060b6-15da-43c7-a85c-e964e92085ee None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:37.741277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61060b6-15da-43c7-a85c-e964e92085ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:37.756273 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-355d446d-7695-4b29-b3fc-94dbae110a7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '387bb82a80fc434eb46a424c7c544762', 'request_id': 'req-355d446d-7695-4b29-b3fc-94dbae110a7d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:37.798489 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-355d446d-7695-4b29-b3fc-94dbae110a7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '387bb82a80fc434eb46a424c7c544762', 'request_id': 'req-355d446d-7695-4b29-b3fc-94dbae110a7d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:37.810379 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e61060b6-15da-43c7-a85c-e964e92085ee None None] MFA Rules not processed for user `aa5417789b27472bb39d85a08253100b`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:37.811179 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-355d446d-7695-4b29-b3fc-94dbae110a7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '387bb82a80fc434eb46a424c7c544762', 'request_id': 'req-355d446d-7695-4b29-b3fc-94dbae110a7d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:37.813799 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e61060b6-15da-43c7-a85c-e964e92085ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:37.840938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-355d446d-7695-4b29-b3fc-94dbae110a7d admin admin] The token cache is being invalidate because project 387bb82a80fc434eb46a424c7c544762 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:37.841235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-355d446d-7695-4b29-b3fc-94dbae110a7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:37.843832 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-355d446d-7695-4b29-b3fc-94dbae110a7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:37.846311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 997/2011] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:37 2022] DELETE /identity/v3/projects/387bb82a80fc434eb46a424c7c544762 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:37.851070 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c307f0-ab22-4244-8123-570de5ec491f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:37.851534 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c307f0-ab22-4244-8123-570de5ec491f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:37.851966 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c307f0-ab22-4244-8123-570de5ec491f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:37.872819 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a1c307f0-ab22-4244-8123-570de5ec491f None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:37.874724 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a1c307f0-ab22-4244-8123-570de5ec491f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:37.908377 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1015/2012] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:29:37.912925 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61060b6-15da-43c7-a85c-e964e92085ee None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:37.915366 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e61060b6-15da-43c7-a85c-e964e92085ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:37.924856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:37.958233 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 998/2013] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:37.959904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f480e4d-8e90-4bea-9784-cbc9947c3b73 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:37.960197 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f480e4d-8e90-4bea-9784-cbc9947c3b73 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:37.960474 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f480e4d-8e90-4bea-9784-cbc9947c3b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:37.978563 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6f480e4d-8e90-4bea-9784-cbc9947c3b73 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:37.979769 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6f480e4d-8e90-4bea-9784-cbc9947c3b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:37.987433 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61060b6-15da-43c7-a85c-e964e92085ee None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:37.989061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 999/2014] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:37.993079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4274abb-1933-42c6-b514-5d4ad9c0a262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:37.993276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f480e4d-8e90-4bea-9784-cbc9947c3b73 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:37.994447 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4274abb-1933-42c6-b514-5d4ad9c0a262 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:37.994896 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4274abb-1933-42c6-b514-5d4ad9c0a262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:37.995352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4274abb-1933-42c6-b514-5d4ad9c0a262 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:37.996221 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4274abb-1933-42c6-b514-5d4ad9c0a262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:37.999150 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4274abb-1933-42c6-b514-5d4ad9c0a262 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:38.013359 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f480e4d-8e90-4bea-9784-cbc9947c3b73 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:38.014373 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:38.025303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a868923-c12c-40d3-bc76-db5c1114a958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:38.025998 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a868923-c12c-40d3-bc76-db5c1114a958 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:38.026801 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a868923-c12c-40d3-bc76-db5c1114a958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:38.027091 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a868923-c12c-40d3-bc76-db5c1114a958 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:38.027831 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a868923-c12c-40d3-bc76-db5c1114a958 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:29:38.028140 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a868923-c12c-40d3-bc76-db5c1114a958 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:38.029920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a868923-c12c-40d3-bc76-db5c1114a958 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:38.082105 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1000/2015] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:29:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:29:38.088281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a868923-c12c-40d3-bc76-db5c1114a958 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:38.095051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1016/2016] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:37 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:38.095051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4274abb-1933-42c6-b514-5d4ad9c0a262 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:38.103622 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a868923-c12c-40d3-bc76-db5c1114a958 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:38.204870 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0a175-6cb5-49e1-919f-13384c3aadd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:38.205736 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0a175-6cb5-49e1-919f-13384c3aadd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:38.206538 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0a175-6cb5-49e1-919f-13384c3aadd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:38.206824 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0a175-6cb5-49e1-919f-13384c3aadd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:38.207530 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0a175-6cb5-49e1-919f-13384c3aadd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:38.210556 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0a175-6cb5-49e1-919f-13384c3aadd0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:38.265855 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1001/2017] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:38 2022] POST /identity/v3/projects => generated 387 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:38.268436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0a175-6cb5-49e1-919f-13384c3aadd0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:38.342169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4274abb-1933-42c6-b514-5d4ad9c0a262 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:38.471726 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0a175-6cb5-49e1-919f-13384c3aadd0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:38.485161 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32163315-19e7-4736-8288-a3fd38a94027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:38.486110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32163315-19e7-4736-8288-a3fd38a94027 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:38.486551 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32163315-19e7-4736-8288-a3fd38a94027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:38.486900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32163315-19e7-4736-8288-a3fd38a94027 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:38.487710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32163315-19e7-4736-8288-a3fd38a94027 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:38.490079 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32163315-19e7-4736-8288-a3fd38a94027 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:38.525518 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e6d49-173d-4e51-a13e-e5475d14cd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:38.525760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1002/2018] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:38 2022] POST /identity/v3/users => generated 294 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:38.526041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e6d49-173d-4e51-a13e-e5475d14cd57 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:38.526393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e6d49-173d-4e51-a13e-e5475d14cd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:38.526678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e6d49-173d-4e51-a13e-e5475d14cd57 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:38.527374 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e6d49-173d-4e51-a13e-e5475d14cd57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:38.530027 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e6d49-173d-4e51-a13e-e5475d14cd57 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:38.530162 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32163315-19e7-4736-8288-a3fd38a94027 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:38.551141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32163315-19e7-4736-8288-a3fd38a94027 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:38.557688 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d5f7d-2ef1-4a00-b0f9-8c6c96084c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:38.558386 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d5f7d-2ef1-4a00-b0f9-8c6c96084c0b admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:38.558695 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d5f7d-2ef1-4a00-b0f9-8c6c96084c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:38.558914 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d5f7d-2ef1-4a00-b0f9-8c6c96084c0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:38.559680 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d5f7d-2ef1-4a00-b0f9-8c6c96084c0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:38.562503 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d5f7d-2ef1-4a00-b0f9-8c6c96084c0b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:38.565549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1017/2019] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:38 2022] POST /identity/v3/users => generated 299 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:38.570238 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e6d49-173d-4e51-a13e-e5475d14cd57 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:38.581316 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1003/2020] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:38.582412 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d5f7d-2ef1-4a00-b0f9-8c6c96084c0b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:38.594611 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e6d49-173d-4e51-a13e-e5475d14cd57 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:38.608206 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d5f7d-2ef1-4a00-b0f9-8c6c96084c0b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:38.608617 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44d920-7058-4d98-8c25-c82a6c063635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:38.609314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44d920-7058-4d98-8c25-c82a6c063635 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:38.609628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44d920-7058-4d98-8c25-c82a6c063635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:38.610652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44d920-7058-4d98-8c25-c82a6c063635 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:38.611395 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44d920-7058-4d98-8c25-c82a6c063635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:38.613038 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44d920-7058-4d98-8c25-c82a6c063635 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:38.621448 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ea168-7c54-44c6-97d7-a74075f76d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:38.622152 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ea168-7c54-44c6-97d7-a74075f76d27 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:38.622437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ea168-7c54-44c6-97d7-a74075f76d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:38.622714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ea168-7c54-44c6-97d7-a74075f76d27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:38.623601 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2ea168-7c54-44c6-97d7-a74075f76d27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:38.630155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2ea168-7c54-44c6-97d7-a74075f76d27 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:38.657754 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1018/2021] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:38.662622 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44d920-7058-4d98-8c25-c82a6c063635 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:38.671227 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1004/2022] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:38.672652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ea168-7c54-44c6-97d7-a74075f76d27 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:38.705516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44d920-7058-4d98-8c25-c82a6c063635 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:38.713973 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969951a8-d182-4e78-9c6b-99ff16b99643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:38.715019 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969951a8-d182-4e78-9c6b-99ff16b99643 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:38.715392 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969951a8-d182-4e78-9c6b-99ff16b99643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:38.715786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969951a8-d182-4e78-9c6b-99ff16b99643 admin admin] PATH_INFO: `/v3/projects/6a4eaf483f4c4b4694d1a0817d7b1077/users/381c17fa45104a20906c3a5b6e0a92f7/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:38.717463 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ea168-7c54-44c6-97d7-a74075f76d27 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:38.734546 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df1a5f-342e-4685-9dc5-715cb69c9ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:38.734844 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969951a8-d182-4e78-9c6b-99ff16b99643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a4eaf483f4c4b4694d1a0817d7b1077, user_id=381c17fa45104a20906c3a5b6e0a92f7, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:38.735460 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df1a5f-342e-4685-9dc5-715cb69c9ca9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:38.735750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df1a5f-342e-4685-9dc5-715cb69c9ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:38.738613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969951a8-d182-4e78-9c6b-99ff16b99643 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:38.738766 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df1a5f-342e-4685-9dc5-715cb69c9ca9 admin admin] PATH_INFO: `/v3/projects/aa3e52705a214f7187c28cede15b7c83/users/670c8c44197549c9b47c06c5f8fd3c12/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:38.768189 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:38.770595 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1019/2023] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:38 2022] PUT /identity/v3/projects/6a4eaf483f4c4b4694d1a0817d7b1077/users/381c17fa45104a20906c3a5b6e0a92f7/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:38.774796 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969951a8-d182-4e78-9c6b-99ff16b99643 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:38.780291 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16df1a5f-342e-4685-9dc5-715cb69c9ca9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa3e52705a214f7187c28cede15b7c83, user_id=670c8c44197549c9b47c06c5f8fd3c12, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:38.783207 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16df1a5f-342e-4685-9dc5-715cb69c9ca9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:38.819322 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1005/2024] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:38 2022] PUT /identity/v3/projects/aa3e52705a214f7187c28cede15b7c83/users/670c8c44197549c9b47c06c5f8fd3c12/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:38.836117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab880012-9855-46d3-99c8-1014357f7239 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:38.836419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab880012-9855-46d3-99c8-1014357f7239 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:38.836702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab880012-9855-46d3-99c8-1014357f7239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:38.869422 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:38.873993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ab880012-9855-46d3-99c8-1014357f7239 None None] MFA Rules not processed for user `670c8c44197549c9b47c06c5f8fd3c12`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:38.875422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ab880012-9855-46d3-99c8-1014357f7239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:38.883051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969951a8-d182-4e78-9c6b-99ff16b99643 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:38.962125 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1006/2025] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:29:38.965995 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab880012-9855-46d3-99c8-1014357f7239 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:38.977744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3bd4b-5ae3-4c64-9ddd-8bd732ad0c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:38.978486 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3bd4b-5ae3-4c64-9ddd-8bd732ad0c6c admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:38.978781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3bd4b-5ae3-4c64-9ddd-8bd732ad0c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:38.979069 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3bd4b-5ae3-4c64-9ddd-8bd732ad0c6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:38.979809 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3bd4b-5ae3-4c64-9ddd-8bd732ad0c6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:38.987039 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3bd4b-5ae3-4c64-9ddd-8bd732ad0c6c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:39.012577 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1020/2026] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:29:39.015199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3bd4b-5ae3-4c64-9ddd-8bd732ad0c6c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:39.034820 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3bd4b-5ae3-4c64-9ddd-8bd732ad0c6c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:39.041441 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f0068-6556-4cda-8fa2-41726f8b10e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:39.042684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f0068-6556-4cda-8fa2-41726f8b10e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:39.043141 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f0068-6556-4cda-8fa2-41726f8b10e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:39.043557 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f0068-6556-4cda-8fa2-41726f8b10e5 admin admin] PATH_INFO: `/v3/projects/6a4eaf483f4c4b4694d1a0817d7b1077/users/381c17fa45104a20906c3a5b6e0a92f7/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:39.046546 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f0068-6556-4cda-8fa2-41726f8b10e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a4eaf483f4c4b4694d1a0817d7b1077, user_id=381c17fa45104a20906c3a5b6e0a92f7, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:39.049373 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f0068-6556-4cda-8fa2-41726f8b10e5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:39.053561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab880012-9855-46d3-99c8-1014357f7239 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:39.059062 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116f9cc-955e-4a16-89fb-0f08437abc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:39.059326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116f9cc-955e-4a16-89fb-0f08437abc62 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:39.059611 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116f9cc-955e-4a16-89fb-0f08437abc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:39.059891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116f9cc-955e-4a16-89fb-0f08437abc62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:39.061858 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116f9cc-955e-4a16-89fb-0f08437abc62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:39.062896 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1021/2027] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:39 2022] PUT /identity/v3/projects/6a4eaf483f4c4b4694d1a0817d7b1077/users/381c17fa45104a20906c3a5b6e0a92f7/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:39.064699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116f9cc-955e-4a16-89fb-0f08437abc62 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:39.079654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4c486-a152-4e49-ba55-c8b794e8f267 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:39.080028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4c486-a152-4e49-ba55-c8b794e8f267 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:39.080792 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4c486-a152-4e49-ba55-c8b794e8f267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:39.101417 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1007/2028] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:38 2022] POST /identity/v3/projects => generated 387 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:39.108613 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116f9cc-955e-4a16-89fb-0f08437abc62 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:39.140470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-00d4c486-a152-4e49-ba55-c8b794e8f267 None None] MFA Rules not processed for user `381c17fa45104a20906c3a5b6e0a92f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:39.141741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-00d4c486-a152-4e49-ba55-c8b794e8f267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:39.189845 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116f9cc-955e-4a16-89fb-0f08437abc62 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:39.194974 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076e559-c7f5-4459-9e61-098a54050068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:39.195817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076e559-c7f5-4459-9e61-098a54050068 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:39.196111 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076e559-c7f5-4459-9e61-098a54050068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:39.196542 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076e559-c7f5-4459-9e61-098a54050068 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:39.197249 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076e559-c7f5-4459-9e61-098a54050068 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:39.200529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076e559-c7f5-4459-9e61-098a54050068 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:39.211326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:39.231290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1008/2029] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:39 2022] POST /identity/v3/users => generated 294 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:39.232572 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076e559-c7f5-4459-9e61-098a54050068 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:39.242862 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1022/2030] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:39.244370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076e559-c7f5-4459-9e61-098a54050068 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:39.251466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ec565-546d-4d00-b4d4-8d710d9e2816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:39.252107 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ec565-546d-4d00-b4d4-8d710d9e2816 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:39.252400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ec565-546d-4d00-b4d4-8d710d9e2816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:39.252690 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ec565-546d-4d00-b4d4-8d710d9e2816 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:39.253057 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4c486-a152-4e49-ba55-c8b794e8f267 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:39.253430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298ec565-546d-4d00-b4d4-8d710d9e2816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:39.255080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298ec565-546d-4d00-b4d4-8d710d9e2816 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:39.255786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-00d4c486-a152-4e49-ba55-c8b794e8f267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:39.267378 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1009/2031] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:39.277296 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ec565-546d-4d00-b4d4-8d710d9e2816 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:39.289562 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ec565-546d-4d00-b4d4-8d710d9e2816 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:39.295875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23400fb-3830-496d-be1f-0fc0c4e5e0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:39.296890 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23400fb-3830-496d-be1f-0fc0c4e5e0a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:39.297138 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23400fb-3830-496d-be1f-0fc0c4e5e0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:39.297422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23400fb-3830-496d-be1f-0fc0c4e5e0a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:39.298349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23400fb-3830-496d-be1f-0fc0c4e5e0a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:39.300215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23400fb-3830-496d-be1f-0fc0c4e5e0a5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:39.316718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1010/2032] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:39.317798 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23400fb-3830-496d-be1f-0fc0c4e5e0a5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:39.329644 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23400fb-3830-496d-be1f-0fc0c4e5e0a5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:39.338722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b15702-cf2e-4c4f-906d-0c28c3cd23d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:39.339724 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b15702-cf2e-4c4f-906d-0c28c3cd23d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:39.340082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b15702-cf2e-4c4f-906d-0c28c3cd23d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:39.340429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b15702-cf2e-4c4f-906d-0c28c3cd23d4 admin admin] PATH_INFO: `/v3/projects/f2337de75c82423b8a62699075a7aca4/users/3f2c591b95d14541baea8649a9918924/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:39.340640 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4c486-a152-4e49-ba55-c8b794e8f267 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:39.346110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f2316-863c-4104-9efb-50e02736a760 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:39.346914 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f2316-863c-4104-9efb-50e02736a760 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:39.347209 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f2316-863c-4104-9efb-50e02736a760 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:39.347491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f2316-863c-4104-9efb-50e02736a760 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:39.360666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f2316-863c-4104-9efb-50e02736a760 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:39.362350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f2316-863c-4104-9efb-50e02736a760 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:39.372853 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b15702-cf2e-4c4f-906d-0c28c3cd23d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2337de75c82423b8a62699075a7aca4, user_id=3f2c591b95d14541baea8649a9918924, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:39.374849 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1023/2033] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:29:39.377469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b15702-cf2e-4c4f-906d-0c28c3cd23d4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:39.399570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1011/2034] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:39 2022] PUT /identity/v3/projects/f2337de75c82423b8a62699075a7aca4/users/3f2c591b95d14541baea8649a9918924/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:39.418350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303a7cb-3bf5-4fe3-95f7-32bce55a574d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:39.418638 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303a7cb-3bf5-4fe3-95f7-32bce55a574d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:39.418918 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303a7cb-3bf5-4fe3-95f7-32bce55a574d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:39.451017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:39.463707 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6303a7cb-3bf5-4fe3-95f7-32bce55a574d None None] MFA Rules not processed for user `3f2c591b95d14541baea8649a9918924`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:39.465566 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6303a7cb-3bf5-4fe3-95f7-32bce55a574d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:39.575335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1024/2035] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:39.584627 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b15702-cf2e-4c4f-906d-0c28c3cd23d4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:39.679005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b15702-cf2e-4c4f-906d-0c28c3cd23d4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:39.698086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9b8cc-527a-49f8-ac06-7a9e19541ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:39.698869 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9b8cc-527a-49f8-ac06-7a9e19541ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:39.699155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9b8cc-527a-49f8-ac06-7a9e19541ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:39.699435 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9b8cc-527a-49f8-ac06-7a9e19541ed5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:39.700139 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a9b8cc-527a-49f8-ac06-7a9e19541ed5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:39.702955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a9b8cc-527a-49f8-ac06-7a9e19541ed5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:39.744281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1012/2036] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:39 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:39.749281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303a7cb-3bf5-4fe3-95f7-32bce55a574d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:39.762019 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303a7cb-3bf5-4fe3-95f7-32bce55a574d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:39.918266 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5886502-eba7-4106-b947-4b118574875a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:39.919120 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5886502-eba7-4106-b947-4b118574875a admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:39.919477 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5886502-eba7-4106-b947-4b118574875a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:39.919829 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5886502-eba7-4106-b947-4b118574875a admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:39.921389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5886502-eba7-4106-b947-4b118574875a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:39.923570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5886502-eba7-4106-b947-4b118574875a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:39.949424 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1025/2037] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:39 2022] POST /identity/v3/users => generated 300 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:29:39.953749 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9b8cc-527a-49f8-ac06-7a9e19541ed5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:39.966573 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9b8cc-527a-49f8-ac06-7a9e19541ed5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:39.974204 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a183a-583e-4f4c-b244-20890c7a5049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:39.975021 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a183a-583e-4f4c-b244-20890c7a5049 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:39.975470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a183a-583e-4f4c-b244-20890c7a5049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:39.975923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a183a-583e-4f4c-b244-20890c7a5049 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:39.976784 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a183a-583e-4f4c-b244-20890c7a5049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:39.987220 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a183a-583e-4f4c-b244-20890c7a5049 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:40.004854 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1013/2038] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:29:40.009299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5886502-eba7-4106-b947-4b118574875a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:40.025286 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5886502-eba7-4106-b947-4b118574875a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:40.044605 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae60386-9aa2-4ed7-a6b8-24c11fcf9262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:40.049956 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae60386-9aa2-4ed7-a6b8-24c11fcf9262 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:40.051012 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae60386-9aa2-4ed7-a6b8-24c11fcf9262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:40.051729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae60386-9aa2-4ed7-a6b8-24c11fcf9262 admin admin] PATH_INFO: `/v3/projects/9102fb174e3642a9899319278aa152b2/users/b2e4275c299a44f68e0f8075342942bf/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:40.086422 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae60386-9aa2-4ed7-a6b8-24c11fcf9262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9102fb174e3642a9899319278aa152b2, user_id=b2e4275c299a44f68e0f8075342942bf, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:40.090648 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae60386-9aa2-4ed7-a6b8-24c11fcf9262 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:40.120019 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1026/2039] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:40 2022] PUT /identity/v3/projects/9102fb174e3642a9899319278aa152b2/users/b2e4275c299a44f68e0f8075342942bf/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:40.124615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a183a-583e-4f4c-b244-20890c7a5049 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:40.173557 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a183a-583e-4f4c-b244-20890c7a5049 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:40.283857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71812776-8680-4a7a-97e7-eaed9c60d5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:40.283857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71812776-8680-4a7a-97e7-eaed9c60d5b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:40.283857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71812776-8680-4a7a-97e7-eaed9c60d5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:40.283857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71812776-8680-4a7a-97e7-eaed9c60d5b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:40.284340 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71812776-8680-4a7a-97e7-eaed9c60d5b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:40.285401 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71812776-8680-4a7a-97e7-eaed9c60d5b1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:40.299027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1014/2040] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:40.303014 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae60386-9aa2-4ed7-a6b8-24c11fcf9262 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:40.318158 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71812776-8680-4a7a-97e7-eaed9c60d5b1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:40.318363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae60386-9aa2-4ed7-a6b8-24c11fcf9262 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:40.323220 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5b5d9-92dd-4fac-b753-aab74a235651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:40.324119 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5b5d9-92dd-4fac-b753-aab74a235651 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:40.324410 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5b5d9-92dd-4fac-b753-aab74a235651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:40.324693 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5b5d9-92dd-4fac-b753-aab74a235651 admin admin] PATH_INFO: `/v3/projects/9102fb174e3642a9899319278aa152b2/users/b2e4275c299a44f68e0f8075342942bf/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:40.327899 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5b5d9-92dd-4fac-b753-aab74a235651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9102fb174e3642a9899319278aa152b2, user_id=b2e4275c299a44f68e0f8075342942bf, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:40.330281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5b5d9-92dd-4fac-b753-aab74a235651 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:40.331245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71812776-8680-4a7a-97e7-eaed9c60d5b1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:40.336762 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa411089-58a0-4d48-aa54-c14f9afebc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:40.337713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa411089-58a0-4d48-aa54-c14f9afebc27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:40.338005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa411089-58a0-4d48-aa54-c14f9afebc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:40.338337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa411089-58a0-4d48-aa54-c14f9afebc27 admin admin] PATH_INFO: `/v3/users/aa5417789b27472bb39d85a08253100b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:40.339782 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa411089-58a0-4d48-aa54-c14f9afebc27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa5417789b27472bb39d85a08253100b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:40.341607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1027/2041] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:40 2022] PUT /identity/v3/projects/9102fb174e3642a9899319278aa152b2/users/b2e4275c299a44f68e0f8075342942bf/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:40.342311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa411089-58a0-4d48-aa54-c14f9afebc27 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:40.352969 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9fe33-4cf2-411b-b04c-b6bfd346b392 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:40.353263 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9fe33-4cf2-411b-b04c-b6bfd346b392 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:40.353546 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9fe33-4cf2-411b-b04c-b6bfd346b392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:40.359305 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:40.386770 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2ff9fe33-4cf2-411b-b04c-b6bfd346b392 None None] MFA Rules not processed for user `b2e4275c299a44f68e0f8075342942bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:40.388104 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2ff9fe33-4cf2-411b-b04c-b6bfd346b392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:40.402583 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aa411089-58a0-4d48-aa54-c14f9afebc27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa5417789b27472bb39d85a08253100b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:40.403908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aa411089-58a0-4d48-aa54-c14f9afebc27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa5417789b27472bb39d85a08253100b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:40.418643 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aa411089-58a0-4d48-aa54-c14f9afebc27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa5417789b27472bb39d85a08253100b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:40.427900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aa411089-58a0-4d48-aa54-c14f9afebc27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa5417789b27472bb39d85a08253100b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:40.428596 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-aa411089-58a0-4d48-aa54-c14f9afebc27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa5417789b27472bb39d85a08253100b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:40.435957 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:40.446502 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1015/2042] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:40 2022] DELETE /identity/v3/users/aa5417789b27472bb39d85a08253100b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:40.452887 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500342a-9969-4892-b809-2c565cc72c3a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:40.453192 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500342a-9969-4892-b809-2c565cc72c3a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:40.453712 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500342a-9969-4892-b809-2c565cc72c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:40.474772 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6500342a-9969-4892-b809-2c565cc72c3a None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:40.476022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6500342a-9969-4892-b809-2c565cc72c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:40.484736 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1028/2043] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:29:40.497039 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9fe33-4cf2-411b-b04c-b6bfd346b392 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:40.499625 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2ff9fe33-4cf2-411b-b04c-b6bfd346b392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:40.562218 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1016/2044] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:29:40.572270 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9fe33-4cf2-411b-b04c-b6bfd346b392 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:40.573063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6500342a-9969-4892-b809-2c565cc72c3a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:40.574596 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6500342a-9969-4892-b809-2c565cc72c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:40.634048 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12050419-1ff2-4a0d-b77e-f688126056ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:40.635478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12050419-1ff2-4a0d-b77e-f688126056ca service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:40.635946 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12050419-1ff2-4a0d-b77e-f688126056ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:40.636915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12050419-1ff2-4a0d-b77e-f688126056ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:40.661209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6500342a-9969-4892-b809-2c565cc72c3a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:40.666212 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2b148-d7d9-4274-9949-59dcb37397e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:40.666872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2b148-d7d9-4274-9949-59dcb37397e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:40.667499 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2b148-d7d9-4274-9949-59dcb37397e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:40.667499 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2b148-d7d9-4274-9949-59dcb37397e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:40.677739 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2b148-d7d9-4274-9949-59dcb37397e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:40.679215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2b148-d7d9-4274-9949-59dcb37397e5 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:40.691793 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1017/2045] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:29:40.725814 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12050419-1ff2-4a0d-b77e-f688126056ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:40.727744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12050419-1ff2-4a0d-b77e-f688126056ca service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:40.759916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1029/2046] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:29:41.606617 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2b148-d7d9-4274-9949-59dcb37397e5 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:41.609372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-eca2b148-d7d9-4274-9949-59dcb37397e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:41.631495 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2b148-d7d9-4274-9949-59dcb37397e5 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:41.663170 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23969de-192a-40c2-b3be-0370dccebf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:41.665676 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23969de-192a-40c2-b3be-0370dccebf00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:41.669809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23969de-192a-40c2-b3be-0370dccebf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:41.671959 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23969de-192a-40c2-b3be-0370dccebf00 admin admin] PATH_INFO: `/v3/projects/e775a32af8b64595bbd5cb1d2f1545f0` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:41.676586 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23969de-192a-40c2-b3be-0370dccebf00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e775a32af8b64595bbd5cb1d2f1545f0)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:41.684351 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23969de-192a-40c2-b3be-0370dccebf00 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:41.722166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12050419-1ff2-4a0d-b77e-f688126056ca service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:41.725071 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-12050419-1ff2-4a0d-b77e-f688126056ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:41.764402 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12050419-1ff2-4a0d-b77e-f688126056ca service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:41.789749 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbcc7e-b2fd-4c83-b8b6-a1f1c07925a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:41.790734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbcc7e-b2fd-4c83-b8b6-a1f1c07925a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:41.791024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbcc7e-b2fd-4c83-b8b6-a1f1c07925a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:41.791305 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbcc7e-b2fd-4c83-b8b6-a1f1c07925a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:41.793667 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-71dbcc7e-b2fd-4c83-b8b6-a1f1c07925a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:41.808594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:41.841148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e23969de-192a-40c2-b3be-0370dccebf00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e775a32af8b64595bbd5cb1d2f1545f0', 'request_id': 'req-e23969de-192a-40c2-b3be-0370dccebf00'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:41.851523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:41.859720 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e23969de-192a-40c2-b3be-0370dccebf00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e775a32af8b64595bbd5cb1d2f1545f0', 'request_id': 'req-e23969de-192a-40c2-b3be-0370dccebf00'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:41.874787 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbcc7e-b2fd-4c83-b8b6-a1f1c07925a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:41.876008 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e23969de-192a-40c2-b3be-0370dccebf00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e775a32af8b64595bbd5cb1d2f1545f0', 'request_id': 'req-e23969de-192a-40c2-b3be-0370dccebf00'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:41.876458 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbcc7e-b2fd-4c83-b8b6-a1f1c07925a8 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:41.900929 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1030/2047] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:29:41.914711 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e23969de-192a-40c2-b3be-0370dccebf00 admin admin] The token cache is being invalidate because project e775a32af8b64595bbd5cb1d2f1545f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:41.915007 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e23969de-192a-40c2-b3be-0370dccebf00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:41.916946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e23969de-192a-40c2-b3be-0370dccebf00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:41.923458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1018/2048] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:41 2022] DELETE /identity/v3/projects/e775a32af8b64595bbd5cb1d2f1545f0 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:41.927318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbcc7e-b2fd-4c83-b8b6-a1f1c07925a8 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:41.953179 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbcc7e-b2fd-4c83-b8b6-a1f1c07925a8 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:42.070031 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cca16f-1d4e-4599-8ab8-4c0e235dee07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:42.071144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cca16f-1d4e-4599-8ab8-4c0e235dee07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:42.072078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cca16f-1d4e-4599-8ab8-4c0e235dee07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:42.072554 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cca16f-1d4e-4599-8ab8-4c0e235dee07 admin admin] PATH_INFO: `/v3/users/381c17fa45104a20906c3a5b6e0a92f7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:42.076010 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cca16f-1d4e-4599-8ab8-4c0e235dee07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=381c17fa45104a20906c3a5b6e0a92f7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:42.078853 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cca16f-1d4e-4599-8ab8-4c0e235dee07 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:42.235663 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-81cca16f-1d4e-4599-8ab8-4c0e235dee07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '381c17fa45104a20906c3a5b6e0a92f7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:42.276701 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-81cca16f-1d4e-4599-8ab8-4c0e235dee07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '381c17fa45104a20906c3a5b6e0a92f7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:42.277493 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-81cca16f-1d4e-4599-8ab8-4c0e235dee07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '381c17fa45104a20906c3a5b6e0a92f7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:42.290844 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-81cca16f-1d4e-4599-8ab8-4c0e235dee07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '381c17fa45104a20906c3a5b6e0a92f7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:42.291602 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-81cca16f-1d4e-4599-8ab8-4c0e235dee07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '381c17fa45104a20906c3a5b6e0a92f7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:42.323391 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1031/2049] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:41 2022] DELETE /identity/v3/users/381c17fa45104a20906c3a5b6e0a92f7 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:42.802811 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23969de-192a-40c2-b3be-0370dccebf00 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:42.804473 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e23969de-192a-40c2-b3be-0370dccebf00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:42.819083 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23969de-192a-40c2-b3be-0370dccebf00 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:42.889222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572bea5-fd99-4f56-91f0-3097efbef320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:42.890401 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572bea5-fd99-4f56-91f0-3097efbef320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:42.890888 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572bea5-fd99-4f56-91f0-3097efbef320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:42.891357 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572bea5-fd99-4f56-91f0-3097efbef320 admin admin] PATH_INFO: `/v3/projects/6a4eaf483f4c4b4694d1a0817d7b1077` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:42.892682 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572bea5-fd99-4f56-91f0-3097efbef320 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a4eaf483f4c4b4694d1a0817d7b1077)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:42.894723 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572bea5-fd99-4f56-91f0-3097efbef320 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:42.952457 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9572bea5-fd99-4f56-91f0-3097efbef320 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a4eaf483f4c4b4694d1a0817d7b1077', 'request_id': 'req-9572bea5-fd99-4f56-91f0-3097efbef320'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:42.959194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9572bea5-fd99-4f56-91f0-3097efbef320 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a4eaf483f4c4b4694d1a0817d7b1077', 'request_id': 'req-9572bea5-fd99-4f56-91f0-3097efbef320'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:42.964820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9572bea5-fd99-4f56-91f0-3097efbef320 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a4eaf483f4c4b4694d1a0817d7b1077', 'request_id': 'req-9572bea5-fd99-4f56-91f0-3097efbef320'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:42.979263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9572bea5-fd99-4f56-91f0-3097efbef320 admin admin] The token cache is being invalidate because project 6a4eaf483f4c4b4694d1a0817d7b1077 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:42.979547 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9572bea5-fd99-4f56-91f0-3097efbef320 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:42.980159 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9572bea5-fd99-4f56-91f0-3097efbef320 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:42.982138 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1019/2050] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:42 2022] DELETE /identity/v3/projects/6a4eaf483f4c4b4694d1a0817d7b1077 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:42.986688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394658b-8e2b-4f4b-93e1-f01c9a1436f7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:42.987214 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394658b-8e2b-4f4b-93e1-f01c9a1436f7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:42.987314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394658b-8e2b-4f4b-93e1-f01c9a1436f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:43.007759 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1394658b-8e2b-4f4b-93e1-f01c9a1436f7 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:43.009837 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1394658b-8e2b-4f4b-93e1-f01c9a1436f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:43.097329 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1032/2051] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:43.105130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0043d-53a3-4f06-a1de-8bba09931137 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:43.105418 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0043d-53a3-4f06-a1de-8bba09931137 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:43.105736 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0043d-53a3-4f06-a1de-8bba09931137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:43.124448 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-aba0043d-53a3-4f06-a1de-8bba09931137 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:43.125696 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-aba0043d-53a3-4f06-a1de-8bba09931137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:43.133638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1020/2052] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:43.138083 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394658b-8e2b-4f4b-93e1-f01c9a1436f7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:43.143438 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:43.149939 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394658b-8e2b-4f4b-93e1-f01c9a1436f7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:43.155730 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89ea31-abd7-4f5e-8900-ad4597ccfcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:43.156465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89ea31-abd7-4f5e-8900-ad4597ccfcbe admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:43.156817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89ea31-abd7-4f5e-8900-ad4597ccfcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:43.157160 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89ea31-abd7-4f5e-8900-ad4597ccfcbe admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:43.158001 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89ea31-abd7-4f5e-8900-ad4597ccfcbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:29:43.158376 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89ea31-abd7-4f5e-8900-ad4597ccfcbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:43.160043 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89ea31-abd7-4f5e-8900-ad4597ccfcbe admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:43.176661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1033/2053] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:29:43.179728 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0043d-53a3-4f06-a1de-8bba09931137 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:43.192870 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0043d-53a3-4f06-a1de-8bba09931137 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:43.198242 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cfe94-7e52-4e36-a7e5-91a20305dfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:43.199222 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cfe94-7e52-4e36-a7e5-91a20305dfa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:43.199506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cfe94-7e52-4e36-a7e5-91a20305dfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:43.199780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cfe94-7e52-4e36-a7e5-91a20305dfa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:43.200471 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98cfe94-7e52-4e36-a7e5-91a20305dfa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:43.203115 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98cfe94-7e52-4e36-a7e5-91a20305dfa7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:43.246760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1021/2054] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:43 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:43.251438 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89ea31-abd7-4f5e-8900-ad4597ccfcbe admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:43.261629 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89ea31-abd7-4f5e-8900-ad4597ccfcbe admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:43.322082 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de33b93-3e62-4b27-aa79-1254a09fd4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:43.322744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de33b93-3e62-4b27-aa79-1254a09fd4db admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:43.323033 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de33b93-3e62-4b27-aa79-1254a09fd4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:43.323316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de33b93-3e62-4b27-aa79-1254a09fd4db admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:43.324013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de33b93-3e62-4b27-aa79-1254a09fd4db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:43.325978 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de33b93-3e62-4b27-aa79-1254a09fd4db admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:43.347585 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1034/2055] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:43 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:43.351725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cfe94-7e52-4e36-a7e5-91a20305dfa7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:43.364555 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cfe94-7e52-4e36-a7e5-91a20305dfa7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:43.369240 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc596f11-2cbb-4f4c-834f-b5be90f75701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:43.369939 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc596f11-2cbb-4f4c-834f-b5be90f75701 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:43.370225 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc596f11-2cbb-4f4c-834f-b5be90f75701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:43.370612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc596f11-2cbb-4f4c-834f-b5be90f75701 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:43.371346 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc596f11-2cbb-4f4c-834f-b5be90f75701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:43.372951 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc596f11-2cbb-4f4c-834f-b5be90f75701 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:43.384181 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1022/2056] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:43.388149 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de33b93-3e62-4b27-aa79-1254a09fd4db admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:43.398655 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de33b93-3e62-4b27-aa79-1254a09fd4db admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:43.403489 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9939a026-f9f5-4551-b8af-1eb55b96dac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:43.404148 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939a026-f9f5-4551-b8af-1eb55b96dac4 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:43.404538 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939a026-f9f5-4551-b8af-1eb55b96dac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:43.404865 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939a026-f9f5-4551-b8af-1eb55b96dac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:43.405618 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9939a026-f9f5-4551-b8af-1eb55b96dac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:43.407238 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9939a026-f9f5-4551-b8af-1eb55b96dac4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:43.417143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1035/2057] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:43.421809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc596f11-2cbb-4f4c-834f-b5be90f75701 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:43.431717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc596f11-2cbb-4f4c-834f-b5be90f75701 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:43.437119 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f6611-c500-4d07-993a-c47aa97057d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:43.438015 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f6611-c500-4d07-993a-c47aa97057d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:43.438553 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f6611-c500-4d07-993a-c47aa97057d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:43.438831 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f6611-c500-4d07-993a-c47aa97057d2 admin admin] PATH_INFO: `/v3/projects/b08051d2330943a88c0404d5cf2e63ec/users/06bf9514617244949d9871805b6e6e75/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:43.441909 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:43.455012 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f6611-c500-4d07-993a-c47aa97057d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b08051d2330943a88c0404d5cf2e63ec, user_id=06bf9514617244949d9871805b6e6e75, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:43.457745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f6611-c500-4d07-993a-c47aa97057d2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:43.470251 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1023/2058] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:43 2022] PUT /identity/v3/projects/b08051d2330943a88c0404d5cf2e63ec/users/06bf9514617244949d9871805b6e6e75/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:43.480237 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de82ddf-dc31-486a-92a1-3bf1b26143c3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:43.480538 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de82ddf-dc31-486a-92a1-3bf1b26143c3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:43.480827 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de82ddf-dc31-486a-92a1-3bf1b26143c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:43.513925 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2de82ddf-dc31-486a-92a1-3bf1b26143c3 None None] MFA Rules not processed for user `06bf9514617244949d9871805b6e6e75`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:43.515170 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2de82ddf-dc31-486a-92a1-3bf1b26143c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:43.588298 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1036/2059] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:43.590080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f6611-c500-4d07-993a-c47aa97057d2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:43.664215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f6611-c500-4d07-993a-c47aa97057d2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:43.669317 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163ccce-b1bd-4802-ba66-afe88c5e8f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:43.670648 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163ccce-b1bd-4802-ba66-afe88c5e8f7f admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:43.671002 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163ccce-b1bd-4802-ba66-afe88c5e8f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:43.671282 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163ccce-b1bd-4802-ba66-afe88c5e8f7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:43.671989 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163ccce-b1bd-4802-ba66-afe88c5e8f7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:43.674180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163ccce-b1bd-4802-ba66-afe88c5e8f7f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:43.705966 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1024/2060] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:43 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:43.709788 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de82ddf-dc31-486a-92a1-3bf1b26143c3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:43.722750 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de82ddf-dc31-486a-92a1-3bf1b26143c3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:43.782895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:43.790217 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7e9eb-c3f6-4af3-89e6-e508a87b96d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:43.791017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7e9eb-c3f6-4af3-89e6-e508a87b96d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:43.791362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7e9eb-c3f6-4af3-89e6-e508a87b96d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:43.791652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7e9eb-c3f6-4af3-89e6-e508a87b96d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:43.792547 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb7e9eb-c3f6-4af3-89e6-e508a87b96d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:43.794531 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb7e9eb-c3f6-4af3-89e6-e508a87b96d8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:43.817977 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1037/2061] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:43 2022] POST /identity/v3/users => generated 299 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:43.821925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163ccce-b1bd-4802-ba66-afe88c5e8f7f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:43.832808 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163ccce-b1bd-4802-ba66-afe88c5e8f7f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:43.837407 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35f946-18ac-4630-902e-95fbb416c731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:43.838093 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35f946-18ac-4630-902e-95fbb416c731 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:43.838382 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35f946-18ac-4630-902e-95fbb416c731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:43.838661 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35f946-18ac-4630-902e-95fbb416c731 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:43.839390 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c35f946-18ac-4630-902e-95fbb416c731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:43.840989 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c35f946-18ac-4630-902e-95fbb416c731 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:43.852740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1025/2062] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:43.856718 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7e9eb-c3f6-4af3-89e6-e508a87b96d8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:43.867052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7e9eb-c3f6-4af3-89e6-e508a87b96d8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:43.871698 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23af26-52f0-4a4c-ba4a-6a3adf095184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:43.872559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23af26-52f0-4a4c-ba4a-6a3adf095184 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:43.872845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23af26-52f0-4a4c-ba4a-6a3adf095184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:43.873653 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23af26-52f0-4a4c-ba4a-6a3adf095184 admin admin] PATH_INFO: `/v3/projects/fbc4d94744cd4bddb6edf2f2946ff85f/users/79b560215baf4d639d4875e07276f630/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:43.889620 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23af26-52f0-4a4c-ba4a-6a3adf095184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbc4d94744cd4bddb6edf2f2946ff85f, user_id=79b560215baf4d639d4875e07276f630, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:43.893098 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23af26-52f0-4a4c-ba4a-6a3adf095184 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:43.909800 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1038/2063] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:43 2022] PUT /identity/v3/projects/fbc4d94744cd4bddb6edf2f2946ff85f/users/79b560215baf4d639d4875e07276f630/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:43.913682 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35f946-18ac-4630-902e-95fbb416c731 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:43.925422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35f946-18ac-4630-902e-95fbb416c731 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:43.981967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ce937-ff54-417b-a96e-6bba117bd55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:43.982637 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ce937-ff54-417b-a96e-6bba117bd55b admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:43.982921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ce937-ff54-417b-a96e-6bba117bd55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:43.983200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ce937-ff54-417b-a96e-6bba117bd55b admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:43.983976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ce937-ff54-417b-a96e-6bba117bd55b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:43.985572 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ce937-ff54-417b-a96e-6bba117bd55b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:43.995655 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1026/2064] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:44.003881 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23af26-52f0-4a4c-ba4a-6a3adf095184 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:44.015933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23af26-52f0-4a4c-ba4a-6a3adf095184 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:44.020694 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3a568-559a-45cd-8970-4a8d787824ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:44.021647 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3a568-559a-45cd-8970-4a8d787824ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:44.022151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3a568-559a-45cd-8970-4a8d787824ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:44.022437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3a568-559a-45cd-8970-4a8d787824ec admin admin] PATH_INFO: `/v3/projects/fbc4d94744cd4bddb6edf2f2946ff85f/users/79b560215baf4d639d4875e07276f630/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:44.026165 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def3a568-559a-45cd-8970-4a8d787824ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbc4d94744cd4bddb6edf2f2946ff85f, user_id=79b560215baf4d639d4875e07276f630, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:44.031960 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def3a568-559a-45cd-8970-4a8d787824ec admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:44.041241 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1039/2065] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:44 2022] PUT /identity/v3/projects/fbc4d94744cd4bddb6edf2f2946ff85f/users/79b560215baf4d639d4875e07276f630/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:44.053676 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd7ae1-f07a-4fe0-95d5-b07200e1d567 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:44.053934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd7ae1-f07a-4fe0-95d5-b07200e1d567 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:44.054220 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd7ae1-f07a-4fe0-95d5-b07200e1d567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:44.087075 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-12fd7ae1-f07a-4fe0-95d5-b07200e1d567 None None] MFA Rules not processed for user `79b560215baf4d639d4875e07276f630`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:44.088314 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-12fd7ae1-f07a-4fe0-95d5-b07200e1d567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:44.143036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:44.169156 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1027/2066] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:44.180401 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3a568-559a-45cd-8970-4a8d787824ec admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:44.182674 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-def3a568-559a-45cd-8970-4a8d787824ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:44.250113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3a568-559a-45cd-8970-4a8d787824ec admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:44.254636 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877bebc6-cda5-4a0e-8933-4f0add58265a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:44.255320 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bebc6-cda5-4a0e-8933-4f0add58265a service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:44.255610 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bebc6-cda5-4a0e-8933-4f0add58265a service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:44.255896 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bebc6-cda5-4a0e-8933-4f0add58265a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:44.320057 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877bebc6-cda5-4a0e-8933-4f0add58265a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:44.321861 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877bebc6-cda5-4a0e-8933-4f0add58265a service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:44.331985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1040/2067] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:46.400203 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881c924-bdc7-4b90-a018-de6dfadd3e4f None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:46.400703 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881c924-bdc7-4b90-a018-de6dfadd3e4f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:46.400895 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881c924-bdc7-4b90-a018-de6dfadd3e4f None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:46.402444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1028/2068] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 30 18:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:29:46.412674 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a0fdf-09b2-4d77-8d3f-1447270a66b6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:46.413135 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a0fdf-09b2-4d77-8d3f-1447270a66b6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:46.413298 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a0fdf-09b2-4d77-8d3f-1447270a66b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:46.431611 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-722a0fdf-09b2-4d77-8d3f-1447270a66b6 None None] MFA Rules not processed for user `49325c1733174e0c98b3475a10c9b5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:46.432819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-722a0fdf-09b2-4d77-8d3f-1447270a66b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:46.505816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:46.518574 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1041/2069] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 30 18:29:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:46.530109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881c924-bdc7-4b90-a018-de6dfadd3e4f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:46.531892 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1881c924-bdc7-4b90-a018-de6dfadd3e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:46.549338 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881c924-bdc7-4b90-a018-de6dfadd3e4f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:46.554774 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc2d4e-e4df-4fcb-94fa-8b0075e0f671 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49325c1733174e0c98b3475a10c9b5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:46.555404 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc2d4e-e4df-4fcb-94fa-8b0075e0f671 service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:46.555713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc2d4e-e4df-4fcb-94fa-8b0075e0f671 service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:46.555987 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc2d4e-e4df-4fcb-94fa-8b0075e0f671 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:46.565791 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc2d4e-e4df-4fcb-94fa-8b0075e0f671 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:46.567381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc2d4e-e4df-4fcb-94fa-8b0075e0f671 service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:46.577116 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1029/2070] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:46.984181 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a0fdf-09b2-4d77-8d3f-1447270a66b6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:46.985673 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-722a0fdf-09b2-4d77-8d3f-1447270a66b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:47.089586 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a0fdf-09b2-4d77-8d3f-1447270a66b6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:47.095818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc091db-8a98-4d06-9956-2f5d294a0f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:47.096737 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc091db-8a98-4d06-9956-2f5d294a0f82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:47.097048 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc091db-8a98-4d06-9956-2f5d294a0f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:47.097341 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc091db-8a98-4d06-9956-2f5d294a0f82 admin admin] PATH_INFO: `/v3/users/44b95c5ee1374995b9d2d09cf662bb30` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:47.098920 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc091db-8a98-4d06-9956-2f5d294a0f82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44b95c5ee1374995b9d2d09cf662bb30)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:47.100998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc091db-8a98-4d06-9956-2f5d294a0f82 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:47.177797 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5cc091db-8a98-4d06-9956-2f5d294a0f82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44b95c5ee1374995b9d2d09cf662bb30'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:47.193912 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5cc091db-8a98-4d06-9956-2f5d294a0f82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44b95c5ee1374995b9d2d09cf662bb30'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:47.194826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5cc091db-8a98-4d06-9956-2f5d294a0f82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44b95c5ee1374995b9d2d09cf662bb30'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:47.204991 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5cc091db-8a98-4d06-9956-2f5d294a0f82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44b95c5ee1374995b9d2d09cf662bb30'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:47.205723 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5cc091db-8a98-4d06-9956-2f5d294a0f82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44b95c5ee1374995b9d2d09cf662bb30'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:47.230810 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1042/2071] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:46 2022] DELETE /identity/v3/users/44b95c5ee1374995b9d2d09cf662bb30 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:47.508288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc2d4e-e4df-4fcb-94fa-8b0075e0f671 service placement] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:47.531389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc2d4e-e4df-4fcb-94fa-8b0075e0f671 service placement] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:47.594220 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78160f9a-b85f-408a-b200-22f6e62416b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:47.595875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78160f9a-b85f-408a-b200-22f6e62416b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:47.596172 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78160f9a-b85f-408a-b200-22f6e62416b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:47.596482 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78160f9a-b85f-408a-b200-22f6e62416b0 admin admin] PATH_INFO: `/v3/projects/92d388c9d937446d90a47eeb70eb54c4` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:47.597790 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78160f9a-b85f-408a-b200-22f6e62416b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92d388c9d937446d90a47eeb70eb54c4)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:47.599799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78160f9a-b85f-408a-b200-22f6e62416b0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:47.667159 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-78160f9a-b85f-408a-b200-22f6e62416b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92d388c9d937446d90a47eeb70eb54c4', 'request_id': 'req-78160f9a-b85f-408a-b200-22f6e62416b0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:47.675334 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-78160f9a-b85f-408a-b200-22f6e62416b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92d388c9d937446d90a47eeb70eb54c4', 'request_id': 'req-78160f9a-b85f-408a-b200-22f6e62416b0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:47.681051 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-78160f9a-b85f-408a-b200-22f6e62416b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92d388c9d937446d90a47eeb70eb54c4', 'request_id': 'req-78160f9a-b85f-408a-b200-22f6e62416b0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:47.695390 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-78160f9a-b85f-408a-b200-22f6e62416b0 admin admin] The token cache is being invalidate because project 92d388c9d937446d90a47eeb70eb54c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:47.695681 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-78160f9a-b85f-408a-b200-22f6e62416b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:47.696487 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-78160f9a-b85f-408a-b200-22f6e62416b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:47.698316 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1030/2072] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:47 2022] DELETE /identity/v3/projects/92d388c9d937446d90a47eeb70eb54c4 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:47.702150 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc091db-8a98-4d06-9956-2f5d294a0f82 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:47.703560 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5cc091db-8a98-4d06-9956-2f5d294a0f82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:47.717096 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc091db-8a98-4d06-9956-2f5d294a0f82 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:47.774989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22041086-b43a-40f1-b9ab-e2d6177376d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:47.775861 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22041086-b43a-40f1-b9ab-e2d6177376d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:47.776148 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22041086-b43a-40f1-b9ab-e2d6177376d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:47.776431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22041086-b43a-40f1-b9ab-e2d6177376d7 admin admin] PATH_INFO: `/v3/users/7bff3b7c7bb84e5e8baaca8e23450b28` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:47.777670 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22041086-b43a-40f1-b9ab-e2d6177376d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bff3b7c7bb84e5e8baaca8e23450b28)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:47.780126 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22041086-b43a-40f1-b9ab-e2d6177376d7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:47.802614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:47.829140 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-22041086-b43a-40f1-b9ab-e2d6177376d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bff3b7c7bb84e5e8baaca8e23450b28'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:47.841164 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-22041086-b43a-40f1-b9ab-e2d6177376d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bff3b7c7bb84e5e8baaca8e23450b28'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:47.842452 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-22041086-b43a-40f1-b9ab-e2d6177376d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bff3b7c7bb84e5e8baaca8e23450b28'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:47.849208 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-22041086-b43a-40f1-b9ab-e2d6177376d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bff3b7c7bb84e5e8baaca8e23450b28'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:47.849879 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-22041086-b43a-40f1-b9ab-e2d6177376d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bff3b7c7bb84e5e8baaca8e23450b28'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:47.862858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1043/2073] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:47 2022] DELETE /identity/v3/users/7bff3b7c7bb84e5e8baaca8e23450b28 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:48.038944 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78160f9a-b85f-408a-b200-22f6e62416b0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:48.040477 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-78160f9a-b85f-408a-b200-22f6e62416b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:48.055168 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78160f9a-b85f-408a-b200-22f6e62416b0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:48.119982 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:48.127721 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34bfdc-b9b0-4ecd-be85-142ce9b0331d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:48.128773 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34bfdc-b9b0-4ecd-be85-142ce9b0331d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:48.129092 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34bfdc-b9b0-4ecd-be85-142ce9b0331d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:48.129427 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34bfdc-b9b0-4ecd-be85-142ce9b0331d admin admin] PATH_INFO: `/v3/projects/73706b4b9b3441b6ad1cb77ec25f3e71` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:48.130766 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf34bfdc-b9b0-4ecd-be85-142ce9b0331d admin admin] RBAC: Authorizing `identity:delete_project(project_id=73706b4b9b3441b6ad1cb77ec25f3e71)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:48.133052 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf34bfdc-b9b0-4ecd-be85-142ce9b0331d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:48.197931 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bf34bfdc-b9b0-4ecd-be85-142ce9b0331d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73706b4b9b3441b6ad1cb77ec25f3e71', 'request_id': 'req-bf34bfdc-b9b0-4ecd-be85-142ce9b0331d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:48.205834 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bf34bfdc-b9b0-4ecd-be85-142ce9b0331d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73706b4b9b3441b6ad1cb77ec25f3e71', 'request_id': 'req-bf34bfdc-b9b0-4ecd-be85-142ce9b0331d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:48.213992 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bf34bfdc-b9b0-4ecd-be85-142ce9b0331d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73706b4b9b3441b6ad1cb77ec25f3e71', 'request_id': 'req-bf34bfdc-b9b0-4ecd-be85-142ce9b0331d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:48.229177 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bf34bfdc-b9b0-4ecd-be85-142ce9b0331d admin admin] The token cache is being invalidate because project 73706b4b9b3441b6ad1cb77ec25f3e71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:48.229389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bf34bfdc-b9b0-4ecd-be85-142ce9b0331d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:48.230374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bf34bfdc-b9b0-4ecd-be85-142ce9b0331d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:48.232175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1031/2074] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:48 2022] DELETE /identity/v3/projects/73706b4b9b3441b6ad1cb77ec25f3e71 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:48.238086 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541aaf47-685c-4155-817d-49b5f7b6ac94 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:48.238198 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541aaf47-685c-4155-817d-49b5f7b6ac94 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:48.238486 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541aaf47-685c-4155-817d-49b5f7b6ac94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:48.269968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-541aaf47-685c-4155-817d-49b5f7b6ac94 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:48.271422 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-541aaf47-685c-4155-817d-49b5f7b6ac94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:48.353896 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1044/2075] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:48.359724 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086bb29-c5d3-4cea-8831-c5995da205fd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:48.359724 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086bb29-c5d3-4cea-8831-c5995da205fd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:48.360262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086bb29-c5d3-4cea-8831-c5995da205fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:48.383131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1086bb29-c5d3-4cea-8831-c5995da205fd None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:48.384640 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1086bb29-c5d3-4cea-8831-c5995da205fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:48.394958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1032/2076] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:48.408202 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541aaf47-685c-4155-817d-49b5f7b6ac94 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:48.421075 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541aaf47-685c-4155-817d-49b5f7b6ac94 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:48.426139 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5dcf98-a6e4-41b0-868e-5b29b7944878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:48.427035 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5dcf98-a6e4-41b0-868e-5b29b7944878 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:48.427405 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5dcf98-a6e4-41b0-868e-5b29b7944878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:48.428378 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5dcf98-a6e4-41b0-868e-5b29b7944878 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:48.428611 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5dcf98-a6e4-41b0-868e-5b29b7944878 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:29:48.428993 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5dcf98-a6e4-41b0-868e-5b29b7944878 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:48.430734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5dcf98-a6e4-41b0-868e-5b29b7944878 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:48.443354 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1045/2077] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:29:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:29:48.447875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1086bb29-c5d3-4cea-8831-c5995da205fd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:48.462523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1086bb29-c5d3-4cea-8831-c5995da205fd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:48.475975 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1577c-9aa9-4fd8-afaa-876c103640ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:48.477329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1577c-9aa9-4fd8-afaa-876c103640ba admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:48.478004 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1577c-9aa9-4fd8-afaa-876c103640ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:48.478589 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1577c-9aa9-4fd8-afaa-876c103640ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:48.479818 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1577c-9aa9-4fd8-afaa-876c103640ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:48.482292 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1577c-9aa9-4fd8-afaa-876c103640ba admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:48.558256 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1033/2078] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:48 2022] POST /identity/v3/projects => generated 371 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:48.563181 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5dcf98-a6e4-41b0-868e-5b29b7944878 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:48.580846 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5dcf98-a6e4-41b0-868e-5b29b7944878 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:48.704345 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705af6b-033a-465d-8aa2-6ddbccf1a175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:48.705082 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705af6b-033a-465d-8aa2-6ddbccf1a175 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:48.705508 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705af6b-033a-465d-8aa2-6ddbccf1a175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:48.705714 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705af6b-033a-465d-8aa2-6ddbccf1a175 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:48.706403 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705af6b-033a-465d-8aa2-6ddbccf1a175 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:48.708645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705af6b-033a-465d-8aa2-6ddbccf1a175 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:48.744517 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1046/2079] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:48 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:48.748557 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1577c-9aa9-4fd8-afaa-876c103640ba admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:48.768086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1577c-9aa9-4fd8-afaa-876c103640ba admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:48.781973 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b758736-b58d-48ff-9885-aafc4cb6e9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:48.782666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b758736-b58d-48ff-9885-aafc4cb6e9ca admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:48.782957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b758736-b58d-48ff-9885-aafc4cb6e9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:48.783244 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b758736-b58d-48ff-9885-aafc4cb6e9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:48.783987 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b758736-b58d-48ff-9885-aafc4cb6e9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:48.785642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b758736-b58d-48ff-9885-aafc4cb6e9ca admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:48.813020 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705af6b-033a-465d-8aa2-6ddbccf1a175 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:48.821599 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1034/2080] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:29:48.824902 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705af6b-033a-465d-8aa2-6ddbccf1a175 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:48.830945 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9459b6-c580-4148-b564-142b72e69092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:48.831602 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9459b6-c580-4148-b564-142b72e69092 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:48.832566 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9459b6-c580-4148-b564-142b72e69092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:48.833350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9459b6-c580-4148-b564-142b72e69092 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:48.834445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9459b6-c580-4148-b564-142b72e69092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:48.836171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9459b6-c580-4148-b564-142b72e69092 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:48.865815 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1047/2081] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:48.868651 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b758736-b58d-48ff-9885-aafc4cb6e9ca admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:48.883252 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b758736-b58d-48ff-9885-aafc4cb6e9ca admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:48.892230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892257c-a6f1-4042-9feb-13f0ef4c7ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:48.893755 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892257c-a6f1-4042-9feb-13f0ef4c7ffe admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:48.894374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892257c-a6f1-4042-9feb-13f0ef4c7ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:48.894828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892257c-a6f1-4042-9feb-13f0ef4c7ffe admin admin] PATH_INFO: `/v3/projects/a57f569caf9b46a0975129afd497c305/users/fbc42f9f3f124368865c9995402e395c/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:48.924085 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892257c-a6f1-4042-9feb-13f0ef4c7ffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a57f569caf9b46a0975129afd497c305, user_id=fbc42f9f3f124368865c9995402e395c, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:48.926843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892257c-a6f1-4042-9feb-13f0ef4c7ffe admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:48.951570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1035/2082] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:48 2022] PUT /identity/v3/projects/a57f569caf9b46a0975129afd497c305/users/fbc42f9f3f124368865c9995402e395c/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:48.962985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d20015-b073-4f81-b5f9-c9aa47544a9c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:48.962985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d20015-b073-4f81-b5f9-c9aa47544a9c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:48.963205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d20015-b073-4f81-b5f9-c9aa47544a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:48.970706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:48.995044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-19d20015-b073-4f81-b5f9-c9aa47544a9c None None] MFA Rules not processed for user `fbc42f9f3f124368865c9995402e395c`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:48.996894 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-19d20015-b073-4f81-b5f9-c9aa47544a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:49.080891 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1048/2083] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:49.092914 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892257c-a6f1-4042-9feb-13f0ef4c7ffe admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:49.095467 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7892257c-a6f1-4042-9feb-13f0ef4c7ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:49.156694 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892257c-a6f1-4042-9feb-13f0ef4c7ffe admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:49.163894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24004f87-c03f-443a-9933-6925d697af28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:49.164563 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24004f87-c03f-443a-9933-6925d697af28 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:49.164843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24004f87-c03f-443a-9933-6925d697af28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:49.165172 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24004f87-c03f-443a-9933-6925d697af28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:49.175191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24004f87-c03f-443a-9933-6925d697af28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:49.176959 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24004f87-c03f-443a-9933-6925d697af28 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:49.190784 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1036/2084] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:49.646670 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d20015-b073-4f81-b5f9-c9aa47544a9c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:49.656782 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d20015-b073-4f81-b5f9-c9aa47544a9c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:49.660386 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75b115-7346-46e4-8fcc-35b77d9437fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:49.661018 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75b115-7346-46e4-8fcc-35b77d9437fb service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:49.661312 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75b115-7346-46e4-8fcc-35b77d9437fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:49.661620 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75b115-7346-46e4-8fcc-35b77d9437fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:49.663408 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2a75b115-7346-46e4-8fcc-35b77d9437fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:49.723168 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75b115-7346-46e4-8fcc-35b77d9437fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:49.724607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75b115-7346-46e4-8fcc-35b77d9437fb service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:49.733506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1049/2085] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:49.895194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24004f87-c03f-443a-9933-6925d697af28 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:49.896622 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-24004f87-c03f-443a-9933-6925d697af28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:49.909365 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24004f87-c03f-443a-9933-6925d697af28 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:49.913305 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285559a-b656-4ebc-8230-c2ceb366fdf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:49.914599 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285559a-b656-4ebc-8230-c2ceb366fdf4 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:49.914907 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285559a-b656-4ebc-8230-c2ceb366fdf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:49.915186 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285559a-b656-4ebc-8230-c2ceb366fdf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:49.917002 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9285559a-b656-4ebc-8230-c2ceb366fdf4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:49.963382 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:50.010018 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285559a-b656-4ebc-8230-c2ceb366fdf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:50.022233 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285559a-b656-4ebc-8230-c2ceb366fdf4 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:50.037461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1037/2086] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:52.228321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75b115-7346-46e4-8fcc-35b77d9437fb service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:52.298428 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75b115-7346-46e4-8fcc-35b77d9437fb service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:52.304245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96314539-da8d-4578-ac95-e9c6df0e71bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:52.305102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96314539-da8d-4578-ac95-e9c6df0e71bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:52.305384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96314539-da8d-4578-ac95-e9c6df0e71bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:52.305685 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96314539-da8d-4578-ac95-e9c6df0e71bd admin admin] PATH_INFO: `/v3/users/fbc42f9f3f124368865c9995402e395c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:52.306862 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96314539-da8d-4578-ac95-e9c6df0e71bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbc42f9f3f124368865c9995402e395c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:52.308791 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96314539-da8d-4578-ac95-e9c6df0e71bd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:52.347612 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:52.359547 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-96314539-da8d-4578-ac95-e9c6df0e71bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbc42f9f3f124368865c9995402e395c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:52.371519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-96314539-da8d-4578-ac95-e9c6df0e71bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbc42f9f3f124368865c9995402e395c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:52.372196 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-96314539-da8d-4578-ac95-e9c6df0e71bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbc42f9f3f124368865c9995402e395c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:52.380830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-96314539-da8d-4578-ac95-e9c6df0e71bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbc42f9f3f124368865c9995402e395c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:52.381496 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-96314539-da8d-4578-ac95-e9c6df0e71bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbc42f9f3f124368865c9995402e395c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:52.393423 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1050/2087] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:52 2022] DELETE /identity/v3/users/fbc42f9f3f124368865c9995402e395c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:52.398059 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798a547-1415-4883-98fd-57f35b00cb52 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:52.398543 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798a547-1415-4883-98fd-57f35b00cb52 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:52.398543 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798a547-1415-4883-98fd-57f35b00cb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:52.415992 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1798a547-1415-4883-98fd-57f35b00cb52 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:52.417552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1798a547-1415-4883-98fd-57f35b00cb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:52.481255 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1038/2088] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:29:52.492554 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96314539-da8d-4578-ac95-e9c6df0e71bd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:52.494425 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-96314539-da8d-4578-ac95-e9c6df0e71bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:52.558107 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96314539-da8d-4578-ac95-e9c6df0e71bd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:52.562492 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c8c5f-1852-4f40-a703-92f9b26934d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:52.563172 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c8c5f-1852-4f40-a703-92f9b26934d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:52.563462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c8c5f-1852-4f40-a703-92f9b26934d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:52.564012 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c8c5f-1852-4f40-a703-92f9b26934d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:52.573501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c8c5f-1852-4f40-a703-92f9b26934d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:52.574986 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c8c5f-1852-4f40-a703-92f9b26934d5 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:52.585957 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1051/2089] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:53.274801 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798a547-1415-4883-98fd-57f35b00cb52 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:53.276523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1798a547-1415-4883-98fd-57f35b00cb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:53.289079 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798a547-1415-4883-98fd-57f35b00cb52 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:53.293556 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228f43d-4f43-465d-8dd4-21bed887081f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:53.294482 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228f43d-4f43-465d-8dd4-21bed887081f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:53.294749 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228f43d-4f43-465d-8dd4-21bed887081f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:53.295816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228f43d-4f43-465d-8dd4-21bed887081f admin admin] PATH_INFO: `/v3/users/670c8c44197549c9b47c06c5f8fd3c12` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:53.296872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228f43d-4f43-465d-8dd4-21bed887081f admin admin] RBAC: Authorizing `identity:delete_user(user_id=670c8c44197549c9b47c06c5f8fd3c12)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:53.298825 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228f43d-4f43-465d-8dd4-21bed887081f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:53.352538 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1228f43d-4f43-465d-8dd4-21bed887081f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '670c8c44197549c9b47c06c5f8fd3c12'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:53.353217 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1228f43d-4f43-465d-8dd4-21bed887081f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '670c8c44197549c9b47c06c5f8fd3c12'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:53.369039 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1228f43d-4f43-465d-8dd4-21bed887081f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '670c8c44197549c9b47c06c5f8fd3c12'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:53.381031 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c8c5f-1852-4f40-a703-92f9b26934d5 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:53.383078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-794c8c5f-1852-4f40-a703-92f9b26934d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:53.390896 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1228f43d-4f43-465d-8dd4-21bed887081f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '670c8c44197549c9b47c06c5f8fd3c12'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:53.391883 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1228f43d-4f43-465d-8dd4-21bed887081f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '670c8c44197549c9b47c06c5f8fd3c12'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:53.399685 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c8c5f-1852-4f40-a703-92f9b26934d5 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:53.409529 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cae054-bc7e-4a2e-a3cc-273cf5acf539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:53.410768 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cae054-bc7e-4a2e-a3cc-273cf5acf539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:53.411247 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1039/2090] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:53 2022] DELETE /identity/v3/users/670c8c44197549c9b47c06c5f8fd3c12 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:53.411870 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cae054-bc7e-4a2e-a3cc-273cf5acf539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:53.412294 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cae054-bc7e-4a2e-a3cc-273cf5acf539 admin admin] PATH_INFO: `/v3/projects/a57f569caf9b46a0975129afd497c305` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:53.412714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dacf26f-f0c4-4afb-881a-9fa820f892a1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:53.413005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dacf26f-f0c4-4afb-881a-9fa820f892a1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:53.413283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dacf26f-f0c4-4afb-881a-9fa820f892a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:53.413873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cae054-bc7e-4a2e-a3cc-273cf5acf539 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a57f569caf9b46a0975129afd497c305)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:53.416111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cae054-bc7e-4a2e-a3cc-273cf5acf539 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:53.433211 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1dacf26f-f0c4-4afb-881a-9fa820f892a1 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:53.434476 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1dacf26f-f0c4-4afb-881a-9fa820f892a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:53.469372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:53.474544 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c9cae054-bc7e-4a2e-a3cc-273cf5acf539 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a57f569caf9b46a0975129afd497c305', 'request_id': 'req-c9cae054-bc7e-4a2e-a3cc-273cf5acf539'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:53.483161 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c9cae054-bc7e-4a2e-a3cc-273cf5acf539 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a57f569caf9b46a0975129afd497c305', 'request_id': 'req-c9cae054-bc7e-4a2e-a3cc-273cf5acf539'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:53.489185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c9cae054-bc7e-4a2e-a3cc-273cf5acf539 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a57f569caf9b46a0975129afd497c305', 'request_id': 'req-c9cae054-bc7e-4a2e-a3cc-273cf5acf539'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:53.504828 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c9cae054-bc7e-4a2e-a3cc-273cf5acf539 admin admin] The token cache is being invalidate because project a57f569caf9b46a0975129afd497c305 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:53.505123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c9cae054-bc7e-4a2e-a3cc-273cf5acf539 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:53.505797 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c9cae054-bc7e-4a2e-a3cc-273cf5acf539 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:53.507349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1052/2091] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:53 2022] DELETE /identity/v3/projects/a57f569caf9b46a0975129afd497c305 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:53.511973 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7750ee7-d286-427a-aa3a-54e5e0f11102 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:53.512263 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7750ee7-d286-427a-aa3a-54e5e0f11102 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:53.512541 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7750ee7-d286-427a-aa3a-54e5e0f11102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:53.513282 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1040/2092] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:29:53.526908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dacf26f-f0c4-4afb-881a-9fa820f892a1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:53.529256 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1dacf26f-f0c4-4afb-881a-9fa820f892a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:53.539318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b7750ee7-d286-427a-aa3a-54e5e0f11102 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:53.540602 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b7750ee7-d286-427a-aa3a-54e5e0f11102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:53.578995 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:53.599227 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dacf26f-f0c4-4afb-881a-9fa820f892a1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:53.632783 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1053/2093] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:53.651617 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab920a-498f-4d2a-a386-01627a5eb205 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:53.652260 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab920a-498f-4d2a-a386-01627a5eb205 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:53.652545 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab920a-498f-4d2a-a386-01627a5eb205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:53.688008 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-58ab920a-498f-4d2a-a386-01627a5eb205 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:53.689647 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-58ab920a-498f-4d2a-a386-01627a5eb205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:53.704474 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1054/2094] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:53.708679 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab920a-498f-4d2a-a386-01627a5eb205 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:53.710672 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc60b0-4e2a-489b-aaea-4935e775efc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:53.711410 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc60b0-4e2a-489b-aaea-4935e775efc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:53.711776 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc60b0-4e2a-489b-aaea-4935e775efc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:53.712131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc60b0-4e2a-489b-aaea-4935e775efc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:53.714194 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-74bc60b0-4e2a-489b-aaea-4935e775efc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:53.721725 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab920a-498f-4d2a-a386-01627a5eb205 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:53.730535 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc60b0-4e2a-489b-aaea-4935e775efc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:53.731051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae83c0-cc0e-4ad3-b6fd-6ea8d334b9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:53.731796 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae83c0-cc0e-4ad3-b6fd-6ea8d334b9d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:53.732153 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae83c0-cc0e-4ad3-b6fd-6ea8d334b9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:53.732382 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc60b0-4e2a-489b-aaea-4935e775efc7 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:53.732852 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae83c0-cc0e-4ad3-b6fd-6ea8d334b9d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:53.733895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae83c0-cc0e-4ad3-b6fd-6ea8d334b9d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:29:53.734276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae83c0-cc0e-4ad3-b6fd-6ea8d334b9d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:53.735980 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae83c0-cc0e-4ad3-b6fd-6ea8d334b9d3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:53.745973 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1041/2095] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:53.754481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1055/2096] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:29:53.762650 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc60b0-4e2a-489b-aaea-4935e775efc7 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:53.775109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc60b0-4e2a-489b-aaea-4935e775efc7 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:53.782664 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b22d15-99e4-49b5-9855-be0c0e84e15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:53.783338 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b22d15-99e4-49b5-9855-be0c0e84e15e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:53.783674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b22d15-99e4-49b5-9855-be0c0e84e15e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:53.783959 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b22d15-99e4-49b5-9855-be0c0e84e15e admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:53.784709 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b22d15-99e4-49b5-9855-be0c0e84e15e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:53.786800 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b22d15-99e4-49b5-9855-be0c0e84e15e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:53.829889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1042/2097] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:53 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:53.833651 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae83c0-cc0e-4ad3-b6fd-6ea8d334b9d3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:53.861924 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae83c0-cc0e-4ad3-b6fd-6ea8d334b9d3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:53.950367 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25b49c-20f6-444c-bea0-e0b4c602bc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:53.951907 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25b49c-20f6-444c-bea0-e0b4c602bc38 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:53.952203 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25b49c-20f6-444c-bea0-e0b4c602bc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:53.952490 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25b49c-20f6-444c-bea0-e0b4c602bc38 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:53.953213 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e25b49c-20f6-444c-bea0-e0b4c602bc38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:53.955231 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e25b49c-20f6-444c-bea0-e0b4c602bc38 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:54.001001 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1056/2098] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:53 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:54.006106 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b22d15-99e4-49b5-9855-be0c0e84e15e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:54.021543 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b22d15-99e4-49b5-9855-be0c0e84e15e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:54.032563 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9da4df-fea7-42ac-ad05-b4a9d7c1672e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:54.033621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9da4df-fea7-42ac-ad05-b4a9d7c1672e admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:54.034037 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9da4df-fea7-42ac-ad05-b4a9d7c1672e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:54.034453 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9da4df-fea7-42ac-ad05-b4a9d7c1672e admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:54.036056 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9da4df-fea7-42ac-ad05-b4a9d7c1672e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:54.037806 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9da4df-fea7-42ac-ad05-b4a9d7c1672e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:54.054887 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1043/2099] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:54.058940 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25b49c-20f6-444c-bea0-e0b4c602bc38 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:54.073209 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25b49c-20f6-444c-bea0-e0b4c602bc38 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:54.081376 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eaccdf-cc80-4e73-90f3-6471e116dbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:54.081758 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eaccdf-cc80-4e73-90f3-6471e116dbf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:54.082281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eaccdf-cc80-4e73-90f3-6471e116dbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:54.082692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eaccdf-cc80-4e73-90f3-6471e116dbf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:54.083911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eaccdf-cc80-4e73-90f3-6471e116dbf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:54.086376 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eaccdf-cc80-4e73-90f3-6471e116dbf3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:54.112758 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1057/2100] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:54.116545 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9da4df-fea7-42ac-ad05-b4a9d7c1672e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:54.143016 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9da4df-fea7-42ac-ad05-b4a9d7c1672e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:54.148531 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001bcad-b316-439b-952c-0a8ce6847436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:54.149977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001bcad-b316-439b-952c-0a8ce6847436 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:54.150497 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001bcad-b316-439b-952c-0a8ce6847436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:54.150895 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001bcad-b316-439b-952c-0a8ce6847436 admin admin] PATH_INFO: `/v3/projects/5710d16b15c0442ea42d7313f87b887d/users/d6f7d8cf91874e37a87852661d0a3f1a/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:54.190701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b001bcad-b316-439b-952c-0a8ce6847436 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5710d16b15c0442ea42d7313f87b887d, user_id=d6f7d8cf91874e37a87852661d0a3f1a, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:54.195440 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b001bcad-b316-439b-952c-0a8ce6847436 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:54.218955 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eaccdf-cc80-4e73-90f3-6471e116dbf3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:54.223183 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1044/2101] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:54 2022] PUT /identity/v3/projects/5710d16b15c0442ea42d7313f87b887d/users/d6f7d8cf91874e37a87852661d0a3f1a/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:54.225203 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b5eaccdf-cc80-4e73-90f3-6471e116dbf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:54.237199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa755b9-c938-467a-8ec2-dbdffe3232cd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:54.237485 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa755b9-c938-467a-8ec2-dbdffe3232cd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:54.237838 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa755b9-c938-467a-8ec2-dbdffe3232cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:54.247026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eaccdf-cc80-4e73-90f3-6471e116dbf3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:54.293453 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1aa755b9-c938-467a-8ec2-dbdffe3232cd None None] MFA Rules not processed for user `d6f7d8cf91874e37a87852661d0a3f1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:54.297613 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1aa755b9-c938-467a-8ec2-dbdffe3232cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:54.334366 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:54.379445 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfba9af-beec-4c9f-92cc-995c5d1dc2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:54.380331 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfba9af-beec-4c9f-92cc-995c5d1dc2fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:54.380615 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfba9af-beec-4c9f-92cc-995c5d1dc2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:54.381223 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfba9af-beec-4c9f-92cc-995c5d1dc2fc admin admin] PATH_INFO: `/v3/projects/aa3e52705a214f7187c28cede15b7c83` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:54.382714 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfba9af-beec-4c9f-92cc-995c5d1dc2fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa3e52705a214f7187c28cede15b7c83)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:54.384661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfba9af-beec-4c9f-92cc-995c5d1dc2fc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:54.416388 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1045/2102] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:54.429874 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa755b9-c938-467a-8ec2-dbdffe3232cd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:54.432107 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1aa755b9-c938-467a-8ec2-dbdffe3232cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:54.445065 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:54.457329 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5bfba9af-beec-4c9f-92cc-995c5d1dc2fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa3e52705a214f7187c28cede15b7c83', 'request_id': 'req-5bfba9af-beec-4c9f-92cc-995c5d1dc2fc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:54.464955 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5bfba9af-beec-4c9f-92cc-995c5d1dc2fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa3e52705a214f7187c28cede15b7c83', 'request_id': 'req-5bfba9af-beec-4c9f-92cc-995c5d1dc2fc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:54.472828 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5bfba9af-beec-4c9f-92cc-995c5d1dc2fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa3e52705a214f7187c28cede15b7c83', 'request_id': 'req-5bfba9af-beec-4c9f-92cc-995c5d1dc2fc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:54.488914 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5bfba9af-beec-4c9f-92cc-995c5d1dc2fc admin admin] The token cache is being invalidate because project aa3e52705a214f7187c28cede15b7c83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:54.489307 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5bfba9af-beec-4c9f-92cc-995c5d1dc2fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:54.490059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-5bfba9af-beec-4c9f-92cc-995c5d1dc2fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:54.491728 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1058/2103] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:54 2022] DELETE /identity/v3/projects/aa3e52705a214f7187c28cede15b7c83 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:54.496123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfba9af-beec-4c9f-92cc-995c5d1dc2fc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:54.497670 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5bfba9af-beec-4c9f-92cc-995c5d1dc2fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:54.514677 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa755b9-c938-467a-8ec2-dbdffe3232cd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:54.573486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f6878-1dd0-43ee-8077-b6b1a6c2e418 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37a9fea702cd405ea9fc14ec2c689648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:54.574190 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f6878-1dd0-43ee-8077-b6b1a6c2e418 service glance] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:54.574472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f6878-1dd0-43ee-8077-b6b1a6c2e418 service glance] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:54.574753 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f6878-1dd0-43ee-8077-b6b1a6c2e418 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:54.576704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfba9af-beec-4c9f-92cc-995c5d1dc2fc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:54.576820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bd3f6878-1dd0-43ee-8077-b6b1a6c2e418 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:54.581570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190a545-990d-4f14-b3ff-49c2935ae7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:54.582479 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190a545-990d-4f14-b3ff-49c2935ae7c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:54.582775 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190a545-990d-4f14-b3ff-49c2935ae7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:54.583058 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190a545-990d-4f14-b3ff-49c2935ae7c8 admin admin] PATH_INFO: `/v3/users/3f2c591b95d14541baea8649a9918924` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:54.584278 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190a545-990d-4f14-b3ff-49c2935ae7c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f2c591b95d14541baea8649a9918924)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:54.586268 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190a545-990d-4f14-b3ff-49c2935ae7c8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:54.635841 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0190a545-990d-4f14-b3ff-49c2935ae7c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f2c591b95d14541baea8649a9918924'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:54.644181 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f6878-1dd0-43ee-8077-b6b1a6c2e418 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:54.645566 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f6878-1dd0-43ee-8077-b6b1a6c2e418 service glance] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:54.649661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0190a545-990d-4f14-b3ff-49c2935ae7c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f2c591b95d14541baea8649a9918924'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:54.650991 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0190a545-990d-4f14-b3ff-49c2935ae7c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f2c591b95d14541baea8649a9918924'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:54.655744 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1046/2104] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:54.660356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0190a545-990d-4f14-b3ff-49c2935ae7c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f2c591b95d14541baea8649a9918924'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:54.661241 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-0190a545-990d-4f14-b3ff-49c2935ae7c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f2c591b95d14541baea8649a9918924'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:54.677140 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1059/2105] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:54 2022] DELETE /identity/v3/users/3f2c591b95d14541baea8649a9918924 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:55.002600 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f6878-1dd0-43ee-8077-b6b1a6c2e418 service glance] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:55.033632 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f6878-1dd0-43ee-8077-b6b1a6c2e418 service glance] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:55.101531 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:55.138301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f0dea-4672-48be-9b65-f4572e4504b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:55.140391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f0dea-4672-48be-9b65-f4572e4504b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:55.140812 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f0dea-4672-48be-9b65-f4572e4504b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:55.141185 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f0dea-4672-48be-9b65-f4572e4504b6 admin admin] PATH_INFO: `/v3/projects/f2337de75c82423b8a62699075a7aca4` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:55.142952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f0dea-4672-48be-9b65-f4572e4504b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2337de75c82423b8a62699075a7aca4)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:55.145696 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f0dea-4672-48be-9b65-f4572e4504b6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:55.180325 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190a545-990d-4f14-b3ff-49c2935ae7c8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:55.181837 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0190a545-990d-4f14-b3ff-49c2935ae7c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:55.200340 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190a545-990d-4f14-b3ff-49c2935ae7c8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:55.205866 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132bcae-922c-44b7-a02c-6c0404480a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:55.206816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132bcae-922c-44b7-a02c-6c0404480a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:55.207104 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132bcae-922c-44b7-a02c-6c0404480a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:55.207384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132bcae-922c-44b7-a02c-6c0404480a41 admin admin] PATH_INFO: `/v3/users/06bf9514617244949d9871805b6e6e75` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:55.208877 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132bcae-922c-44b7-a02c-6c0404480a41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06bf9514617244949d9871805b6e6e75)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:55.211118 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132bcae-922c-44b7-a02c-6c0404480a41 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:55.227579 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8d2f0dea-4672-48be-9b65-f4572e4504b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2337de75c82423b8a62699075a7aca4', 'request_id': 'req-8d2f0dea-4672-48be-9b65-f4572e4504b6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:55.237391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8d2f0dea-4672-48be-9b65-f4572e4504b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2337de75c82423b8a62699075a7aca4', 'request_id': 'req-8d2f0dea-4672-48be-9b65-f4572e4504b6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:55.245043 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8d2f0dea-4672-48be-9b65-f4572e4504b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2337de75c82423b8a62699075a7aca4', 'request_id': 'req-8d2f0dea-4672-48be-9b65-f4572e4504b6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:55.262089 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8d2f0dea-4672-48be-9b65-f4572e4504b6 admin admin] The token cache is being invalidate because project f2337de75c82423b8a62699075a7aca4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:55.262456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8d2f0dea-4672-48be-9b65-f4572e4504b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:55.263241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8d2f0dea-4672-48be-9b65-f4572e4504b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:55.266000 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1047/2106] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:55 2022] DELETE /identity/v3/projects/f2337de75c82423b8a62699075a7aca4 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:55.269838 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f0dea-4672-48be-9b65-f4572e4504b6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:55.271228 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8d2f0dea-4672-48be-9b65-f4572e4504b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:55.272397 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:55.278430 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d132bcae-922c-44b7-a02c-6c0404480a41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06bf9514617244949d9871805b6e6e75'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:55.293388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d132bcae-922c-44b7-a02c-6c0404480a41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06bf9514617244949d9871805b6e6e75'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:55.294329 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d132bcae-922c-44b7-a02c-6c0404480a41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06bf9514617244949d9871805b6e6e75'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:55.309286 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d132bcae-922c-44b7-a02c-6c0404480a41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06bf9514617244949d9871805b6e6e75'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:55.310358 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d132bcae-922c-44b7-a02c-6c0404480a41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06bf9514617244949d9871805b6e6e75'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:55.333381 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1060/2107] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:55 2022] DELETE /identity/v3/users/06bf9514617244949d9871805b6e6e75 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:55.336230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20471db7-8496-432b-adb0-d9f39b194b6c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:55.336531 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20471db7-8496-432b-adb0-d9f39b194b6c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:55.336820 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20471db7-8496-432b-adb0-d9f39b194b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:55.360078 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f0dea-4672-48be-9b65-f4572e4504b6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:55.360248 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-20471db7-8496-432b-adb0-d9f39b194b6c None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:55.361517 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-20471db7-8496-432b-adb0-d9f39b194b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:55.459011 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44424ca-98d9-4a4b-bc0d-899d7485ab59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:55.459906 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44424ca-98d9-4a4b-bc0d-899d7485ab59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:55.460190 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44424ca-98d9-4a4b-bc0d-899d7485ab59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:55.460472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44424ca-98d9-4a4b-bc0d-899d7485ab59 admin admin] PATH_INFO: `/v3/users/b2e4275c299a44f68e0f8075342942bf` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:55.462275 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44424ca-98d9-4a4b-bc0d-899d7485ab59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2e4275c299a44f68e0f8075342942bf)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:55.464191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44424ca-98d9-4a4b-bc0d-899d7485ab59 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:55.477979 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1061/2108] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:55.487869 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:55.493912 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20471db7-8496-432b-adb0-d9f39b194b6c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:55.495459 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-20471db7-8496-432b-adb0-d9f39b194b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:55.533207 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e44424ca-98d9-4a4b-bc0d-899d7485ab59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2e4275c299a44f68e0f8075342942bf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:55.533957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e44424ca-98d9-4a4b-bc0d-899d7485ab59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2e4275c299a44f68e0f8075342942bf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:55.551720 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e44424ca-98d9-4a4b-bc0d-899d7485ab59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2e4275c299a44f68e0f8075342942bf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:55.562038 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e44424ca-98d9-4a4b-bc0d-899d7485ab59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2e4275c299a44f68e0f8075342942bf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:55.565475 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e44424ca-98d9-4a4b-bc0d-899d7485ab59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2e4275c299a44f68e0f8075342942bf'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:55.583682 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1048/2109] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:55 2022] DELETE /identity/v3/users/b2e4275c299a44f68e0f8075342942bf => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:55.605887 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20471db7-8496-432b-adb0-d9f39b194b6c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:55.673850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c25c8-1793-497d-a27f-3b5fb2b2e33b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:55.674504 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c25c8-1793-497d-a27f-3b5fb2b2e33b service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:55.674790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c25c8-1793-497d-a27f-3b5fb2b2e33b service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:55.675082 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c25c8-1793-497d-a27f-3b5fb2b2e33b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:55.677327 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c09c25c8-1793-497d-a27f-3b5fb2b2e33b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:55.724797 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:55.793080 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c25c8-1793-497d-a27f-3b5fb2b2e33b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:55.794594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c25c8-1793-497d-a27f-3b5fb2b2e33b service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:55.806329 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1062/2110] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:55.977430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44424ca-98d9-4a4b-bc0d-899d7485ab59 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:55.979315 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e44424ca-98d9-4a4b-bc0d-899d7485ab59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:55.993158 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44424ca-98d9-4a4b-bc0d-899d7485ab59 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:55.998939 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a17b0-c799-4dc3-af5c-cfaa3622ec41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:55.999872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a17b0-c799-4dc3-af5c-cfaa3622ec41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:56.000101 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a17b0-c799-4dc3-af5c-cfaa3622ec41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:56.000378 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a17b0-c799-4dc3-af5c-cfaa3622ec41 admin admin] PATH_INFO: `/v3/projects/9102fb174e3642a9899319278aa152b2` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:56.001604 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a17b0-c799-4dc3-af5c-cfaa3622ec41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9102fb174e3642a9899319278aa152b2)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:56.003541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a17b0-c799-4dc3-af5c-cfaa3622ec41 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:56.090614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9c7a17b0-c799-4dc3-af5c-cfaa3622ec41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9102fb174e3642a9899319278aa152b2', 'request_id': 'req-9c7a17b0-c799-4dc3-af5c-cfaa3622ec41'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:56.097825 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9c7a17b0-c799-4dc3-af5c-cfaa3622ec41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9102fb174e3642a9899319278aa152b2', 'request_id': 'req-9c7a17b0-c799-4dc3-af5c-cfaa3622ec41'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:56.104369 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9c7a17b0-c799-4dc3-af5c-cfaa3622ec41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9102fb174e3642a9899319278aa152b2', 'request_id': 'req-9c7a17b0-c799-4dc3-af5c-cfaa3622ec41'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:56.131252 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9c7a17b0-c799-4dc3-af5c-cfaa3622ec41 admin admin] The token cache is being invalidate because project 9102fb174e3642a9899319278aa152b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:56.132983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9c7a17b0-c799-4dc3-af5c-cfaa3622ec41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:56.133893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9c7a17b0-c799-4dc3-af5c-cfaa3622ec41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:56.137792 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1049/2111] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:55 2022] DELETE /identity/v3/projects/9102fb174e3642a9899319278aa152b2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:56.143054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866038f-6657-46e2-98df-5a5f3ac25476 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:56.143631 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866038f-6657-46e2-98df-5a5f3ac25476 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:56.143925 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866038f-6657-46e2-98df-5a5f3ac25476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:56.184440 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7866038f-6657-46e2-98df-5a5f3ac25476 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:56.186261 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7866038f-6657-46e2-98df-5a5f3ac25476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:56.199525 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a17b0-c799-4dc3-af5c-cfaa3622ec41 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:56.200687 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9c7a17b0-c799-4dc3-af5c-cfaa3622ec41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:56.282750 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1063/2112] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:29:56.291273 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed50bff-dd5f-4500-8a04-57620bcb7cc3 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:56.291570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed50bff-dd5f-4500-8a04-57620bcb7cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:56.291854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed50bff-dd5f-4500-8a04-57620bcb7cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:56.293898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a17b0-c799-4dc3-af5c-cfaa3622ec41 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:56.298703 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9949df8-5960-4c2e-9f8a-30e923436818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:56.299556 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9949df8-5960-4c2e-9f8a-30e923436818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:56.299839 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9949df8-5960-4c2e-9f8a-30e923436818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:56.300120 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9949df8-5960-4c2e-9f8a-30e923436818 admin admin] PATH_INFO: `/v3/projects/b08051d2330943a88c0404d5cf2e63ec` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:56.301310 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9949df8-5960-4c2e-9f8a-30e923436818 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b08051d2330943a88c0404d5cf2e63ec)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:56.303276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9949df8-5960-4c2e-9f8a-30e923436818 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:56.310171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2ed50bff-dd5f-4500-8a04-57620bcb7cc3 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:56.311561 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2ed50bff-dd5f-4500-8a04-57620bcb7cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:56.320069 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1064/2113] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:29:56.320711 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:56.324077 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed50bff-dd5f-4500-8a04-57620bcb7cc3 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:56.334482 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed50bff-dd5f-4500-8a04-57620bcb7cc3 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:56.339417 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb110ba7-3e50-48ed-ba61-ea61a4ebd123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:56.340151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb110ba7-3e50-48ed-ba61-ea61a4ebd123 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:56.340440 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb110ba7-3e50-48ed-ba61-ea61a4ebd123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:56.340724 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb110ba7-3e50-48ed-ba61-ea61a4ebd123 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:56.341453 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb110ba7-3e50-48ed-ba61-ea61a4ebd123 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:29:56.341864 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb110ba7-3e50-48ed-ba61-ea61a4ebd123 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:56.344307 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb110ba7-3e50-48ed-ba61-ea61a4ebd123 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:56.354215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e9949df8-5960-4c2e-9f8a-30e923436818 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b08051d2330943a88c0404d5cf2e63ec', 'request_id': 'req-e9949df8-5960-4c2e-9f8a-30e923436818'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:56.356786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1065/2114] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:29:56.361168 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb110ba7-3e50-48ed-ba61-ea61a4ebd123 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:56.361430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e9949df8-5960-4c2e-9f8a-30e923436818 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b08051d2330943a88c0404d5cf2e63ec', 'request_id': 'req-e9949df8-5960-4c2e-9f8a-30e923436818'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:56.367483 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e9949df8-5960-4c2e-9f8a-30e923436818 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b08051d2330943a88c0404d5cf2e63ec', 'request_id': 'req-e9949df8-5960-4c2e-9f8a-30e923436818'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:56.373461 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb110ba7-3e50-48ed-ba61-ea61a4ebd123 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:56.393172 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e9949df8-5960-4c2e-9f8a-30e923436818 admin admin] The token cache is being invalidate because project b08051d2330943a88c0404d5cf2e63ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:56.393521 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e9949df8-5960-4c2e-9f8a-30e923436818 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:56.394143 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e9949df8-5960-4c2e-9f8a-30e923436818 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:56.395959 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1050/2115] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:56 2022] DELETE /identity/v3/projects/b08051d2330943a88c0404d5cf2e63ec => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:56.402470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9949df8-5960-4c2e-9f8a-30e923436818 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:56.403891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e9949df8-5960-4c2e-9f8a-30e923436818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:56.437078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63006c7c-b830-4c79-a370-7813869664a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:56.437769 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63006c7c-b830-4c79-a370-7813869664a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:56.438127 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63006c7c-b830-4c79-a370-7813869664a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:56.438435 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63006c7c-b830-4c79-a370-7813869664a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:56.439592 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63006c7c-b830-4c79-a370-7813869664a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:56.441604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63006c7c-b830-4c79-a370-7813869664a4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:56.471900 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1066/2116] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:56 2022] POST /identity/v3/projects => generated 401 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:29:56.472091 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9949df8-5960-4c2e-9f8a-30e923436818 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:56.472850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63006c7c-b830-4c79-a370-7813869664a4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:56.474496 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-63006c7c-b830-4c79-a370-7813869664a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:56.519638 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:56.535637 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091d224-17b8-4114-9e86-9e501947dbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:56.536515 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091d224-17b8-4114-9e86-9e501947dbda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:56.536841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091d224-17b8-4114-9e86-9e501947dbda admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:56.537184 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091d224-17b8-4114-9e86-9e501947dbda admin admin] PATH_INFO: `/v3/users/79b560215baf4d639d4875e07276f630` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:56.538433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4091d224-17b8-4114-9e86-9e501947dbda admin admin] RBAC: Authorizing `identity:delete_user(user_id=79b560215baf4d639d4875e07276f630)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:56.540373 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4091d224-17b8-4114-9e86-9e501947dbda admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:56.556209 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63006c7c-b830-4c79-a370-7813869664a4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:56.562782 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57ca53-32b6-46a5-95f1-05ed00b65601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:56.563457 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57ca53-32b6-46a5-95f1-05ed00b65601 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:56.563985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57ca53-32b6-46a5-95f1-05ed00b65601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:56.564318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57ca53-32b6-46a5-95f1-05ed00b65601 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:56.565044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57ca53-32b6-46a5-95f1-05ed00b65601 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:56.567265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57ca53-32b6-46a5-95f1-05ed00b65601 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:56.589615 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1067/2117] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:56 2022] POST /identity/v3/users => generated 301 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:29:56.594038 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57ca53-32b6-46a5-95f1-05ed00b65601 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:56.604603 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57ca53-32b6-46a5-95f1-05ed00b65601 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:56.610207 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69de05-ebd5-457b-9f9f-f2ba4f97e1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:56.610902 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69de05-ebd5-457b-9f9f-f2ba4f97e1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:56.611239 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69de05-ebd5-457b-9f9f-f2ba4f97e1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:56.611535 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69de05-ebd5-457b-9f9f-f2ba4f97e1d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:56.612183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba69de05-ebd5-457b-9f9f-f2ba4f97e1d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:56.613925 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba69de05-ebd5-457b-9f9f-f2ba4f97e1d5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:56.616703 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4091d224-17b8-4114-9e86-9e501947dbda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79b560215baf4d639d4875e07276f630'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:56.617388 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4091d224-17b8-4114-9e86-9e501947dbda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79b560215baf4d639d4875e07276f630'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:56.624805 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:56.630438 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1068/2118] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:29:56.631541 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69de05-ebd5-457b-9f9f-f2ba4f97e1d5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:56.634675 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ba69de05-ebd5-457b-9f9f-f2ba4f97e1d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:56.640957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4091d224-17b8-4114-9e86-9e501947dbda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79b560215baf4d639d4875e07276f630'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:56.650201 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69de05-ebd5-457b-9f9f-f2ba4f97e1d5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:56.650359 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4091d224-17b8-4114-9e86-9e501947dbda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79b560215baf4d639d4875e07276f630'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:56.651059 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4091d224-17b8-4114-9e86-9e501947dbda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79b560215baf4d639d4875e07276f630'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:56.655428 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e1d74-47fa-4465-86fa-fc62bb3df982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:56.656811 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e1d74-47fa-4465-86fa-fc62bb3df982 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:56.657105 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e1d74-47fa-4465-86fa-fc62bb3df982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:56.657601 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e1d74-47fa-4465-86fa-fc62bb3df982 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:56.658344 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e1d74-47fa-4465-86fa-fc62bb3df982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:56.660047 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e1d74-47fa-4465-86fa-fc62bb3df982 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:56.673625 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1069/2119] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:56.675197 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e1d74-47fa-4465-86fa-fc62bb3df982 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:56.689953 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1051/2120] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:29:56 2022] DELETE /identity/v3/users/79b560215baf4d639d4875e07276f630 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:56.690136 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e1d74-47fa-4465-86fa-fc62bb3df982 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:56.766249 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cd638-1e62-41e9-843d-fd93adc68116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:56.767902 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cd638-1e62-41e9-843d-fd93adc68116 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:56.768471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cd638-1e62-41e9-843d-fd93adc68116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:56.769001 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cd638-1e62-41e9-843d-fd93adc68116 admin admin] PATH_INFO: `/v3/projects/e66fc5d6ec9d497399aa1ea1510d9c30/users/96be715d1f094ee38d17d0eb8fac6cb1/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:56.791786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1cd638-1e62-41e9-843d-fd93adc68116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e66fc5d6ec9d497399aa1ea1510d9c30, user_id=96be715d1f094ee38d17d0eb8fac6cb1, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:56.796291 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1cd638-1e62-41e9-843d-fd93adc68116 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:56.820124 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1070/2121] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:56 2022] PUT /identity/v3/projects/e66fc5d6ec9d497399aa1ea1510d9c30/users/96be715d1f094ee38d17d0eb8fac6cb1/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:56.833035 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc5ef1-1134-438d-9055-ab5b48e76df6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:56.833699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc5ef1-1134-438d-9055-ab5b48e76df6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:56.834018 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc5ef1-1134-438d-9055-ab5b48e76df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:56.868771 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-abbc5ef1-1134-438d-9055-ab5b48e76df6 None None] MFA Rules not processed for user `96be715d1f094ee38d17d0eb8fac6cb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:56.870197 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-abbc5ef1-1134-438d-9055-ab5b48e76df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:56.962892 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1052/2122] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:56.968494 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cd638-1e62-41e9-843d-fd93adc68116 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:57.055651 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cd638-1e62-41e9-843d-fd93adc68116 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:57.066021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d172cf2-1d7f-44e5-90ce-f64873ccda43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:57.066741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d172cf2-1d7f-44e5-90ce-f64873ccda43 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:57.067060 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d172cf2-1d7f-44e5-90ce-f64873ccda43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:57.067351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d172cf2-1d7f-44e5-90ce-f64873ccda43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:57.068166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d172cf2-1d7f-44e5-90ce-f64873ccda43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:57.070158 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d172cf2-1d7f-44e5-90ce-f64873ccda43 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:57.103918 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1071/2123] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:56 2022] POST /identity/v3/projects => generated 399 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:57.105080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc5ef1-1134-438d-9055-ab5b48e76df6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:57.116815 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc5ef1-1134-438d-9055-ab5b48e76df6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:57.122760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d172cf2-1d7f-44e5-90ce-f64873ccda43 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:57.124496 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5d172cf2-1d7f-44e5-90ce-f64873ccda43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:57.138255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:57.194208 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b78e6-8f02-416d-b31b-94bc99ceaeaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:57.195196 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b78e6-8f02-416d-b31b-94bc99ceaeaa admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:57.195771 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b78e6-8f02-416d-b31b-94bc99ceaeaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:57.195982 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b78e6-8f02-416d-b31b-94bc99ceaeaa admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:57.196708 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b78e6-8f02-416d-b31b-94bc99ceaeaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:57.199024 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b78e6-8f02-416d-b31b-94bc99ceaeaa admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:57.224950 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1053/2124] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:57 2022] POST /identity/v3/users => generated 306 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:57.226550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d172cf2-1d7f-44e5-90ce-f64873ccda43 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:57.231888 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b78e6-8f02-416d-b31b-94bc99ceaeaa admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:57.231982 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f7c6d-9c40-404d-8c59-db50fe69c78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:57.233786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f7c6d-9c40-404d-8c59-db50fe69c78d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:57.233786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f7c6d-9c40-404d-8c59-db50fe69c78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:57.233786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f7c6d-9c40-404d-8c59-db50fe69c78d admin admin] PATH_INFO: `/v3/projects/fbc4d94744cd4bddb6edf2f2946ff85f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:57.234833 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f7c6d-9c40-404d-8c59-db50fe69c78d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbc4d94744cd4bddb6edf2f2946ff85f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:57.236943 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f7c6d-9c40-404d-8c59-db50fe69c78d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:57.261633 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b78e6-8f02-416d-b31b-94bc99ceaeaa admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:57.282427 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ecb7d-562b-4bdd-b3ea-6c84dd633f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:57.283973 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ecb7d-562b-4bdd-b3ea-6c84dd633f3e admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:57.286286 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ecb7d-562b-4bdd-b3ea-6c84dd633f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:57.286777 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ecb7d-562b-4bdd-b3ea-6c84dd633f3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:57.293347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ecb7d-562b-4bdd-b3ea-6c84dd633f3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:57.297516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ecb7d-562b-4bdd-b3ea-6c84dd633f3e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:57.317981 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1054/2125] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:29:57.319448 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ecb7d-562b-4bdd-b3ea-6c84dd633f3e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:57.319972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-911f7c6d-9c40-404d-8c59-db50fe69c78d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbc4d94744cd4bddb6edf2f2946ff85f', 'request_id': 'req-911f7c6d-9c40-404d-8c59-db50fe69c78d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:57.327655 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-911f7c6d-9c40-404d-8c59-db50fe69c78d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbc4d94744cd4bddb6edf2f2946ff85f', 'request_id': 'req-911f7c6d-9c40-404d-8c59-db50fe69c78d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:57.334227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-911f7c6d-9c40-404d-8c59-db50fe69c78d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbc4d94744cd4bddb6edf2f2946ff85f', 'request_id': 'req-911f7c6d-9c40-404d-8c59-db50fe69c78d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:57.334932 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ecb7d-562b-4bdd-b3ea-6c84dd633f3e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:57.350021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-911f7c6d-9c40-404d-8c59-db50fe69c78d admin admin] The token cache is being invalidate because project fbc4d94744cd4bddb6edf2f2946ff85f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:29:57.350509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-911f7c6d-9c40-404d-8c59-db50fe69c78d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:57.351236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-911f7c6d-9c40-404d-8c59-db50fe69c78d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:29:57.353229 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1072/2126] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:29:57 2022] DELETE /identity/v3/projects/fbc4d94744cd4bddb6edf2f2946ff85f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:29:57.358956 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67e4db-7577-4d2e-b054-327c6e01aa18 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:57.359245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67e4db-7577-4d2e-b054-327c6e01aa18 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:57.359632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67e4db-7577-4d2e-b054-327c6e01aa18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:57.382031 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3f67e4db-7577-4d2e-b054-327c6e01aa18 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:57.383393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3f67e4db-7577-4d2e-b054-327c6e01aa18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:57.390623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:57.413056 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb161f97-0c87-4e51-8e9e-a15ef21c5626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:57.414528 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb161f97-0c87-4e51-8e9e-a15ef21c5626 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:57.414528 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb161f97-0c87-4e51-8e9e-a15ef21c5626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:57.414979 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb161f97-0c87-4e51-8e9e-a15ef21c5626 admin admin] PATH_INFO: `/v3/projects/a5dd92b2b227441b80d7cf164b574891/users/b4eae5b010d944abbc29fe2bc1ef2088/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:57.432290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb161f97-0c87-4e51-8e9e-a15ef21c5626 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5dd92b2b227441b80d7cf164b574891, user_id=b4eae5b010d944abbc29fe2bc1ef2088, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:57.435129 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb161f97-0c87-4e51-8e9e-a15ef21c5626 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:57.452010 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1055/2127] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:57 2022] PUT /identity/v3/projects/a5dd92b2b227441b80d7cf164b574891/users/b4eae5b010d944abbc29fe2bc1ef2088/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:57.453007 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1073/2128] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:29:57.455932 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb161f97-0c87-4e51-8e9e-a15ef21c5626 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:57.457837 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-eb161f97-0c87-4e51-8e9e-a15ef21c5626 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:57.461748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731bb117-5fe6-40e9-b4a3-11329e37ee82 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:57.462113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731bb117-5fe6-40e9-b4a3-11329e37ee82 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:57.462462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731bb117-5fe6-40e9-b4a3-11329e37ee82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:57.483206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-731bb117-5fe6-40e9-b4a3-11329e37ee82 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:57.484634 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-731bb117-5fe6-40e9-b4a3-11329e37ee82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:57.532013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:57.545199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb161f97-0c87-4e51-8e9e-a15ef21c5626 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:57.552857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0298d1-f63c-4d62-a523-db07bc52259b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:57.553525 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0298d1-f63c-4d62-a523-db07bc52259b admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:57.553839 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0298d1-f63c-4d62-a523-db07bc52259b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:57.554163 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0298d1-f63c-4d62-a523-db07bc52259b admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:57.554901 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0298d1-f63c-4d62-a523-db07bc52259b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:57.556570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0298d1-f63c-4d62-a523-db07bc52259b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:57.573351 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1056/2129] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:57.574526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1074/2130] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:57.575106 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0298d1-f63c-4d62-a523-db07bc52259b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:57.582977 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731bb117-5fe6-40e9-b4a3-11329e37ee82 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:57.587710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0298d1-f63c-4d62-a523-db07bc52259b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:57.597759 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24af3d8-1812-4347-9d38-26c37932cc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:57.599505 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24af3d8-1812-4347-9d38-26c37932cc41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:57.599858 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24af3d8-1812-4347-9d38-26c37932cc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:57.600264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24af3d8-1812-4347-9d38-26c37932cc41 admin admin] PATH_INFO: `/v3/projects/a5dd92b2b227441b80d7cf164b574891/users/b4eae5b010d944abbc29fe2bc1ef2088/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:57.603018 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24af3d8-1812-4347-9d38-26c37932cc41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5dd92b2b227441b80d7cf164b574891, user_id=b4eae5b010d944abbc29fe2bc1ef2088, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:57.605627 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731bb117-5fe6-40e9-b4a3-11329e37ee82 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:57.606169 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24af3d8-1812-4347-9d38-26c37932cc41 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:57.614403 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f08d7-2eed-461b-93ef-d3a7d44d5bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:57.616739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f08d7-2eed-461b-93ef-d3a7d44d5bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:57.617217 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f08d7-2eed-461b-93ef-d3a7d44d5bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:57.617505 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f08d7-2eed-461b-93ef-d3a7d44d5bc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:57.618616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f08d7-2eed-461b-93ef-d3a7d44d5bc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:29:57.619207 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f08d7-2eed-461b-93ef-d3a7d44d5bc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:57.621560 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f08d7-2eed-461b-93ef-d3a7d44d5bc9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:57.625616 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1057/2131] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:57 2022] PUT /identity/v3/projects/a5dd92b2b227441b80d7cf164b574891/users/b4eae5b010d944abbc29fe2bc1ef2088/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:57.635442 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc71c4-9ffc-47bf-8de6-7c4fd42da8f6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:57.635789 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc71c4-9ffc-47bf-8de6-7c4fd42da8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:57.636014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc71c4-9ffc-47bf-8de6-7c4fd42da8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:57.640923 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1075/2132] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:29:57.641925 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f08d7-2eed-461b-93ef-d3a7d44d5bc9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:57.652382 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f08d7-2eed-461b-93ef-d3a7d44d5bc9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:57.665951 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-67cc71c4-9ffc-47bf-8de6-7c4fd42da8f6 None None] MFA Rules not processed for user `b4eae5b010d944abbc29fe2bc1ef2088`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:57.667457 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-67cc71c4-9ffc-47bf-8de6-7c4fd42da8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:57.733214 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab3b11-e83e-4776-ac27-5fbdd5062952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:57.734181 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab3b11-e83e-4776-ac27-5fbdd5062952 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:57.734575 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab3b11-e83e-4776-ac27-5fbdd5062952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:57.735064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab3b11-e83e-4776-ac27-5fbdd5062952 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:57.735856 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab3b11-e83e-4776-ac27-5fbdd5062952 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:57.738139 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab3b11-e83e-4776-ac27-5fbdd5062952 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:57.747750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1058/2133] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:57 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:57.760196 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc71c4-9ffc-47bf-8de6-7c4fd42da8f6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:57.761847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-67cc71c4-9ffc-47bf-8de6-7c4fd42da8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:57.770525 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1076/2134] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:29:57 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:57.774227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab3b11-e83e-4776-ac27-5fbdd5062952 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:57.799523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:57.828728 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc71c4-9ffc-47bf-8de6-7c4fd42da8f6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:57.869624 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab3b11-e83e-4776-ac27-5fbdd5062952 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:57.878751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8ec18-9380-4670-9fa9-538e524d0ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:57.879614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8ec18-9380-4670-9fa9-538e524d0ebe admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:57.879905 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8ec18-9380-4670-9fa9-538e524d0ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:57.880677 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8ec18-9380-4670-9fa9-538e524d0ebe admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:57.881393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8ec18-9380-4670-9fa9-538e524d0ebe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:57.883858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8ec18-9380-4670-9fa9-538e524d0ebe admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:57.914729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0713d1-1994-451b-b236-a2b7074743ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:57.916013 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0713d1-1994-451b-b236-a2b7074743ff service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:57.916588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0713d1-1994-451b-b236-a2b7074743ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:57.916966 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1077/2135] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:29:57 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:29:57.917383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0713d1-1994-451b-b236-a2b7074743ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:57.921638 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8ec18-9380-4670-9fa9-538e524d0ebe admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:57.934877 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8ec18-9380-4670-9fa9-538e524d0ebe admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:57.943374 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b812837-f8b5-4a04-b26e-54c508cb2d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:57.944437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b812837-f8b5-4a04-b26e-54c508cb2d58 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:57.944796 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b812837-f8b5-4a04-b26e-54c508cb2d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:57.945408 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b812837-f8b5-4a04-b26e-54c508cb2d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:57.946248 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b812837-f8b5-4a04-b26e-54c508cb2d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:57.948068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b812837-f8b5-4a04-b26e-54c508cb2d58 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:57.967865 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1078/2136] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:29:57.972111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b812837-f8b5-4a04-b26e-54c508cb2d58 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:57.984249 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b812837-f8b5-4a04-b26e-54c508cb2d58 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:57.997775 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910a406-0a49-40f0-a1ec-34360c66c5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:57.998668 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910a406-0a49-40f0-a1ec-34360c66c5e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:57.999020 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910a406-0a49-40f0-a1ec-34360c66c5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:57.999608 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910a406-0a49-40f0-a1ec-34360c66c5e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:58.000427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f910a406-0a49-40f0-a1ec-34360c66c5e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:58.002771 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0713d1-1994-451b-b236-a2b7074743ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:58.004312 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0713d1-1994-451b-b236-a2b7074743ff service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:58.006485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f910a406-0a49-40f0-a1ec-34360c66c5e4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:58.019335 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1059/2137] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:57 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:29:58.026488 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1079/2138] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:29:58.031175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0713d1-1994-451b-b236-a2b7074743ff service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:58.042192 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0713d1-1994-451b-b236-a2b7074743ff service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:58.048411 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ff5c7-0148-4737-8715-aebe9340d78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:58.049370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ff5c7-0148-4737-8715-aebe9340d78b admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:58.049596 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ff5c7-0148-4737-8715-aebe9340d78b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:58.049893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ff5c7-0148-4737-8715-aebe9340d78b admin admin] PATH_INFO: `/v3/projects/02241431a7964f609e505e6618133e46/users/6aeea7135604416e962f599e64424173/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:58.069062 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2ff5c7-0148-4737-8715-aebe9340d78b admin admin] RBAC: Authorizing `identity:create_grant(project_id=02241431a7964f609e505e6618133e46, user_id=6aeea7135604416e962f599e64424173, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:58.072304 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2ff5c7-0148-4737-8715-aebe9340d78b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:58.101366 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1060/2139] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:29:58 2022] PUT /identity/v3/projects/02241431a7964f609e505e6618133e46/users/6aeea7135604416e962f599e64424173/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:29:58.114680 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446723b-9794-490a-b2ad-de594838a9c0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:58.115013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446723b-9794-490a-b2ad-de594838a9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:58.115237 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446723b-9794-490a-b2ad-de594838a9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:58.134581 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ff5c7-0148-4737-8715-aebe9340d78b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:58.144398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b446723b-9794-490a-b2ad-de594838a9c0 None None] MFA Rules not processed for user `6aeea7135604416e962f599e64424173`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:58.145696 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b446723b-9794-490a-b2ad-de594838a9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:58.153511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:58.210857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ff5c7-0148-4737-8715-aebe9340d78b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:58.215382 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb095ab-72ba-4a51-97fe-29e16b55f417 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:58.216044 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb095ab-72ba-4a51-97fe-29e16b55f417 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:58.216469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb095ab-72ba-4a51-97fe-29e16b55f417 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:58.216753 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb095ab-72ba-4a51-97fe-29e16b55f417 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:58.219400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-abb095ab-72ba-4a51-97fe-29e16b55f417 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:58.250394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1080/2140] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:29:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:58.250592 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:58.262929 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446723b-9794-490a-b2ad-de594838a9c0 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:58.264741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b446723b-9794-490a-b2ad-de594838a9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:58.277345 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446723b-9794-490a-b2ad-de594838a9c0 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:58.281806 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f17a4f-c96e-4b2e-8f7e-33a5d148b5e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:58.282448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f17a4f-c96e-4b2e-8f7e-33a5d148b5e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:58.282735 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f17a4f-c96e-4b2e-8f7e-33a5d148b5e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:58.283021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f17a4f-c96e-4b2e-8f7e-33a5d148b5e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:58.292277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f17a4f-c96e-4b2e-8f7e-33a5d148b5e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:58.292440 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb095ab-72ba-4a51-97fe-29e16b55f417 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:58.293752 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f17a4f-c96e-4b2e-8f7e-33a5d148b5e3 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:58.293884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb095ab-72ba-4a51-97fe-29e16b55f417 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:58.304576 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1061/2141] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:58 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:29:58.304794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1081/2142] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:30:01.205966 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb095ab-72ba-4a51-97fe-29e16b55f417 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:01.207374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-abb095ab-72ba-4a51-97fe-29e16b55f417 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:01.347673 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb095ab-72ba-4a51-97fe-29e16b55f417 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:01.364863 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7512b-3e1e-44e0-9d61-153477c43490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:01.365729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7512b-3e1e-44e0-9d61-153477c43490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:01.366023 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7512b-3e1e-44e0-9d61-153477c43490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:01.366305 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7512b-3e1e-44e0-9d61-153477c43490 admin admin] PATH_INFO: `/v3/users/db9762ad11e24246aefe2905eafe83f1` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:01.367829 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc7512b-3e1e-44e0-9d61-153477c43490 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db9762ad11e24246aefe2905eafe83f1)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:01.369543 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc7512b-3e1e-44e0-9d61-153477c43490 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:01.430595 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6bc7512b-3e1e-44e0-9d61-153477c43490 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db9762ad11e24246aefe2905eafe83f1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:01.431164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6bc7512b-3e1e-44e0-9d61-153477c43490 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db9762ad11e24246aefe2905eafe83f1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:01.445645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6bc7512b-3e1e-44e0-9d61-153477c43490 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db9762ad11e24246aefe2905eafe83f1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:01.453349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6bc7512b-3e1e-44e0-9d61-153477c43490 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db9762ad11e24246aefe2905eafe83f1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:01.454115 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6bc7512b-3e1e-44e0-9d61-153477c43490 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db9762ad11e24246aefe2905eafe83f1'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:01.466454 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1062/2143] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:30:01 2022] DELETE /identity/v3/users/db9762ad11e24246aefe2905eafe83f1 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:01.660448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f17a4f-c96e-4b2e-8f7e-33a5d148b5e3 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:01.672754 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f17a4f-c96e-4b2e-8f7e-33a5d148b5e3 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:01.731539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770df8e7-4af3-4189-9caf-01c81762756a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:01.732398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770df8e7-4af3-4189-9caf-01c81762756a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:01.732877 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770df8e7-4af3-4189-9caf-01c81762756a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:01.733297 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770df8e7-4af3-4189-9caf-01c81762756a admin admin] PATH_INFO: `/v3/projects/e96b506102ab4528956023c5e0f3dbcf` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:01.734494 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770df8e7-4af3-4189-9caf-01c81762756a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e96b506102ab4528956023c5e0f3dbcf)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:01.736457 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770df8e7-4af3-4189-9caf-01c81762756a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:01.785458 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-770df8e7-4af3-4189-9caf-01c81762756a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e96b506102ab4528956023c5e0f3dbcf', 'request_id': 'req-770df8e7-4af3-4189-9caf-01c81762756a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:01.792125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-770df8e7-4af3-4189-9caf-01c81762756a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e96b506102ab4528956023c5e0f3dbcf', 'request_id': 'req-770df8e7-4af3-4189-9caf-01c81762756a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:01.798238 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-770df8e7-4af3-4189-9caf-01c81762756a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e96b506102ab4528956023c5e0f3dbcf', 'request_id': 'req-770df8e7-4af3-4189-9caf-01c81762756a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:01.811905 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-770df8e7-4af3-4189-9caf-01c81762756a admin admin] The token cache is being invalidate because project e96b506102ab4528956023c5e0f3dbcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:30:01.812331 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-770df8e7-4af3-4189-9caf-01c81762756a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:01.813102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-770df8e7-4af3-4189-9caf-01c81762756a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:01.815197 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1082/2144] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:30:01 2022] DELETE /identity/v3/projects/e96b506102ab4528956023c5e0f3dbcf => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:30:01.820139 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdfee8-62df-4a4a-9dab-a480906108b8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:01.820432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdfee8-62df-4a4a-9dab-a480906108b8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:01.820711 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdfee8-62df-4a4a-9dab-a480906108b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:01.839446 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fafdfee8-62df-4a4a-9dab-a480906108b8 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:01.840681 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fafdfee8-62df-4a4a-9dab-a480906108b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:01.890113 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:01.909231 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1063/2145] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:30:01.917648 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa466ec2-741d-4d38-8b71-9cfa5ff1a41e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:01.917944 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa466ec2-741d-4d38-8b71-9cfa5ff1a41e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:01.918255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa466ec2-741d-4d38-8b71-9cfa5ff1a41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:01.937069 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fa466ec2-741d-4d38-8b71-9cfa5ff1a41e None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:01.938384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fa466ec2-741d-4d38-8b71-9cfa5ff1a41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:01.946836 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1083/2146] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:30:01.950997 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafdfee8-62df-4a4a-9dab-a480906108b8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:01.960667 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafdfee8-62df-4a4a-9dab-a480906108b8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:01.965392 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d7afc-84f7-42c4-b656-f5097b9c2833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:01.966074 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d7afc-84f7-42c4-b656-f5097b9c2833 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:01.966357 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d7afc-84f7-42c4-b656-f5097b9c2833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:01.966637 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d7afc-84f7-42c4-b656-f5097b9c2833 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:01.967958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d7afc-84f7-42c4-b656-f5097b9c2833 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:30:01.968266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d7afc-84f7-42c4-b656-f5097b9c2833 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:01.969879 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d7afc-84f7-42c4-b656-f5097b9c2833 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:01.981550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1064/2147] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:30:01.986502 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa466ec2-741d-4d38-8b71-9cfa5ff1a41e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:01.995760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa466ec2-741d-4d38-8b71-9cfa5ff1a41e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:02.000566 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c1cbb-a51a-4ab7-b9ca-9ba523712fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:02.001320 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c1cbb-a51a-4ab7-b9ca-9ba523712fe1 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:02.001684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c1cbb-a51a-4ab7-b9ca-9ba523712fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:02.002042 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c1cbb-a51a-4ab7-b9ca-9ba523712fe1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:02.002938 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c1cbb-a51a-4ab7-b9ca-9ba523712fe1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:02.004920 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c1cbb-a51a-4ab7-b9ca-9ba523712fe1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:02.018784 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:02.036120 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1084/2148] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:30:01 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:30:02.037329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d7afc-84f7-42c4-b656-f5097b9c2833 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:02.049228 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d7afc-84f7-42c4-b656-f5097b9c2833 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:02.106903 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7132a-4767-4e0e-bb2e-05773fd87ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:02.107567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7132a-4767-4e0e-bb2e-05773fd87ffc admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:02.108740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7132a-4767-4e0e-bb2e-05773fd87ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:02.108740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7132a-4767-4e0e-bb2e-05773fd87ffc admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:02.109315 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7132a-4767-4e0e-bb2e-05773fd87ffc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:02.111288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7132a-4767-4e0e-bb2e-05773fd87ffc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:02.130470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1065/2149] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:30:02 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:30:02.134589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c1cbb-a51a-4ab7-b9ca-9ba523712fe1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:02.145887 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c1cbb-a51a-4ab7-b9ca-9ba523712fe1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:02.150664 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1eb7e-1acc-46ce-b6d9-3bd3a1cd8a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:02.151318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1eb7e-1acc-46ce-b6d9-3bd3a1cd8a61 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:02.151604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1eb7e-1acc-46ce-b6d9-3bd3a1cd8a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:02.151884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1eb7e-1acc-46ce-b6d9-3bd3a1cd8a61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:02.152612 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1eb7e-1acc-46ce-b6d9-3bd3a1cd8a61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:02.154260 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1eb7e-1acc-46ce-b6d9-3bd3a1cd8a61 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:02.166386 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1085/2150] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:02.170785 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7132a-4767-4e0e-bb2e-05773fd87ffc admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:02.180678 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7132a-4767-4e0e-bb2e-05773fd87ffc admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:02.185467 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854172c3-4204-4ebf-95b4-fb5e467b4297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:02.186924 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854172c3-4204-4ebf-95b4-fb5e467b4297 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:02.187298 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854172c3-4204-4ebf-95b4-fb5e467b4297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:02.187577 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854172c3-4204-4ebf-95b4-fb5e467b4297 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:02.188308 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854172c3-4204-4ebf-95b4-fb5e467b4297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:02.190241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854172c3-4204-4ebf-95b4-fb5e467b4297 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:02.201033 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1066/2151] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:02.204684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1eb7e-1acc-46ce-b6d9-3bd3a1cd8a61 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:02.215321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1eb7e-1acc-46ce-b6d9-3bd3a1cd8a61 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:02.219956 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904b8d3-545e-4ff2-925e-ca7dd8d5f0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:02.220831 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904b8d3-545e-4ff2-925e-ca7dd8d5f0e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:02.221111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904b8d3-545e-4ff2-925e-ca7dd8d5f0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:02.221396 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904b8d3-545e-4ff2-925e-ca7dd8d5f0e7 admin admin] PATH_INFO: `/v3/projects/76384f04296b43b69e675de14cf1a55a/users/6ac4fc6cab4f47ea910ffdbed0a8e5b8/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:02.236740 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6904b8d3-545e-4ff2-925e-ca7dd8d5f0e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76384f04296b43b69e675de14cf1a55a, user_id=6ac4fc6cab4f47ea910ffdbed0a8e5b8, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:02.239407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6904b8d3-545e-4ff2-925e-ca7dd8d5f0e7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:02.253886 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1086/2152] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:30:02 2022] PUT /identity/v3/projects/76384f04296b43b69e675de14cf1a55a/users/6ac4fc6cab4f47ea910ffdbed0a8e5b8/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:30:02.258926 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734cacb-373f-438d-9a6f-4c655ec7b4b0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:02.259409 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734cacb-373f-438d-9a6f-4c655ec7b4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:02.259500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734cacb-373f-438d-9a6f-4c655ec7b4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:02.277998 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9734cacb-373f-438d-9a6f-4c655ec7b4b0 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:02.279211 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9734cacb-373f-438d-9a6f-4c655ec7b4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:02.339864 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1067/2153] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:02.349670 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904b8d3-545e-4ff2-925e-ca7dd8d5f0e7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:02.351388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6904b8d3-545e-4ff2-925e-ca7dd8d5f0e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:02.411715 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904b8d3-545e-4ff2-925e-ca7dd8d5f0e7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:02.415500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3a913-1554-4757-bff7-fb3c38d21b11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:02.416142 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3a913-1554-4757-bff7-fb3c38d21b11 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:02.416430 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3a913-1554-4757-bff7-fb3c38d21b11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:02.416829 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3a913-1554-4757-bff7-fb3c38d21b11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:02.427165 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3a913-1554-4757-bff7-fb3c38d21b11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:02.428599 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3a913-1554-4757-bff7-fb3c38d21b11 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:02.438998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1087/2154] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:30:06.559684 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36cfc4-11bd-4518-97e6-299179327e1a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:06.559684 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36cfc4-11bd-4518-97e6-299179327e1a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:06.559684 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36cfc4-11bd-4518-97e6-299179327e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:06.591853 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7c36cfc4-11bd-4518-97e6-299179327e1a None None] MFA Rules not processed for user `6ac4fc6cab4f47ea910ffdbed0a8e5b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:06.593762 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7c36cfc4-11bd-4518-97e6-299179327e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:06.643806 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:06.675953 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1068/2155] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:06.690100 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3a913-1554-4757-bff7-fb3c38d21b11 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:06.691620 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2ac3a913-1554-4757-bff7-fb3c38d21b11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:06.858664 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3a913-1554-4757-bff7-fb3c38d21b11 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:06.870110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9d7a2-af2d-4d8c-8086-c7c29461be91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:06.877823 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9d7a2-af2d-4d8c-8086-c7c29461be91 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:06.900160 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9d7a2-af2d-4d8c-8086-c7c29461be91 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:06.900860 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9d7a2-af2d-4d8c-8086-c7c29461be91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:06.915252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9d7a2-af2d-4d8c-8086-c7c29461be91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:06.917325 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9d7a2-af2d-4d8c-8086-c7c29461be91 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:06.933682 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1088/2156] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:08.705112 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c721d-0aba-476b-acb0-7da48adc0bdc None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:08.705611 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c721d-0aba-476b-acb0-7da48adc0bdc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:08.705922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c721d-0aba-476b-acb0-7da48adc0bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:08.728383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1f2c721d-0aba-476b-acb0-7da48adc0bdc None None] MFA Rules not processed for user `6ac4fc6cab4f47ea910ffdbed0a8e5b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:08.728383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1f2c721d-0aba-476b-acb0-7da48adc0bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:08.737041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1069/2157] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:30:10.288728 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9d7a2-af2d-4d8c-8086-c7c29461be91 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:10.290955 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0fa9d7a2-af2d-4d8c-8086-c7c29461be91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:10.307990 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9d7a2-af2d-4d8c-8086-c7c29461be91 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:10.316924 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f89edb-9231-4b61-8ae4-0f92c95a8401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:10.318864 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f89edb-9231-4b61-8ae4-0f92c95a8401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:10.319343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f89edb-9231-4b61-8ae4-0f92c95a8401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:10.319865 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f89edb-9231-4b61-8ae4-0f92c95a8401 admin admin] PATH_INFO: `/v3/users/96be715d1f094ee38d17d0eb8fac6cb1` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:10.321363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f89edb-9231-4b61-8ae4-0f92c95a8401 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96be715d1f094ee38d17d0eb8fac6cb1)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:10.323824 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f89edb-9231-4b61-8ae4-0f92c95a8401 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:10.414196 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f5f89edb-9231-4b61-8ae4-0f92c95a8401 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96be715d1f094ee38d17d0eb8fac6cb1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:10.415506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:10.448929 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f5f89edb-9231-4b61-8ae4-0f92c95a8401 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96be715d1f094ee38d17d0eb8fac6cb1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:10.449848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f5f89edb-9231-4b61-8ae4-0f92c95a8401 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96be715d1f094ee38d17d0eb8fac6cb1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:10.458198 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f5f89edb-9231-4b61-8ae4-0f92c95a8401 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96be715d1f094ee38d17d0eb8fac6cb1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:10.458998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f5f89edb-9231-4b61-8ae4-0f92c95a8401 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96be715d1f094ee38d17d0eb8fac6cb1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:10.476489 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1089/2158] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:30:10 2022] DELETE /identity/v3/users/96be715d1f094ee38d17d0eb8fac6cb1 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:10.480879 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fa50b-c82a-4536-833a-af1258dd94b2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:10.481810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fa50b-c82a-4536-833a-af1258dd94b2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:10.482092 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fa50b-c82a-4536-833a-af1258dd94b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:10.503349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-960fa50b-c82a-4536-833a-af1258dd94b2 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:10.504760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-960fa50b-c82a-4536-833a-af1258dd94b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:10.594479 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1070/2159] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:10.604377 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f89edb-9231-4b61-8ae4-0f92c95a8401 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:10.605862 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f5f89edb-9231-4b61-8ae4-0f92c95a8401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:10.667529 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f89edb-9231-4b61-8ae4-0f92c95a8401 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:10.672614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf424a-01cc-449d-a442-e272004cb689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:10.673343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf424a-01cc-449d-a442-e272004cb689 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:10.673897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf424a-01cc-449d-a442-e272004cb689 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:10.674079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf424a-01cc-449d-a442-e272004cb689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:10.689931 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf424a-01cc-449d-a442-e272004cb689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:10.691610 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf424a-01cc-449d-a442-e272004cb689 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:10.714478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1090/2160] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:11.061461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960fa50b-c82a-4536-833a-af1258dd94b2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:11.063777 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-960fa50b-c82a-4536-833a-af1258dd94b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:11.081433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960fa50b-c82a-4536-833a-af1258dd94b2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:11.087138 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00685a45-0a00-4719-80c6-2b76944cf830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:11.088584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00685a45-0a00-4719-80c6-2b76944cf830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:11.088584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00685a45-0a00-4719-80c6-2b76944cf830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:11.088809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00685a45-0a00-4719-80c6-2b76944cf830 admin admin] PATH_INFO: `/v3/projects/e66fc5d6ec9d497399aa1ea1510d9c30` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:11.089865 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00685a45-0a00-4719-80c6-2b76944cf830 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e66fc5d6ec9d497399aa1ea1510d9c30)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:11.091808 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00685a45-0a00-4719-80c6-2b76944cf830 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:11.160017 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-00685a45-0a00-4719-80c6-2b76944cf830 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e66fc5d6ec9d497399aa1ea1510d9c30', 'request_id': 'req-00685a45-0a00-4719-80c6-2b76944cf830'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:11.166364 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-00685a45-0a00-4719-80c6-2b76944cf830 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e66fc5d6ec9d497399aa1ea1510d9c30', 'request_id': 'req-00685a45-0a00-4719-80c6-2b76944cf830'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:11.172704 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-00685a45-0a00-4719-80c6-2b76944cf830 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e66fc5d6ec9d497399aa1ea1510d9c30', 'request_id': 'req-00685a45-0a00-4719-80c6-2b76944cf830'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:11.185716 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-00685a45-0a00-4719-80c6-2b76944cf830 admin admin] The token cache is being invalidate because project e66fc5d6ec9d497399aa1ea1510d9c30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:30:11.186497 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-00685a45-0a00-4719-80c6-2b76944cf830 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:11.187124 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-00685a45-0a00-4719-80c6-2b76944cf830 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:11.188686 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1071/2161] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:30:11 2022] DELETE /identity/v3/projects/e66fc5d6ec9d497399aa1ea1510d9c30 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:30:11.193036 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf424a-01cc-449d-a442-e272004cb689 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:11.203091 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf424a-01cc-449d-a442-e272004cb689 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:11.267952 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4c967-a685-45c4-912b-a20348bd679a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:11.268822 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4c967-a685-45c4-912b-a20348bd679a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:11.269142 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4c967-a685-45c4-912b-a20348bd679a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:11.269420 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4c967-a685-45c4-912b-a20348bd679a admin admin] PATH_INFO: `/v3/users/b4eae5b010d944abbc29fe2bc1ef2088` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:11.270684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4c967-a685-45c4-912b-a20348bd679a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4eae5b010d944abbc29fe2bc1ef2088)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:11.272643 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4c967-a685-45c4-912b-a20348bd679a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:11.320156 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-22c4c967-a685-45c4-912b-a20348bd679a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4eae5b010d944abbc29fe2bc1ef2088'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:11.335111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-22c4c967-a685-45c4-912b-a20348bd679a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4eae5b010d944abbc29fe2bc1ef2088'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:11.335849 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-22c4c967-a685-45c4-912b-a20348bd679a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4eae5b010d944abbc29fe2bc1ef2088'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:11.337357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:11.350134 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-22c4c967-a685-45c4-912b-a20348bd679a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4eae5b010d944abbc29fe2bc1ef2088'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:11.350985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-22c4c967-a685-45c4-912b-a20348bd679a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4eae5b010d944abbc29fe2bc1ef2088'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:11.367716 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1091/2162] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:30:11 2022] DELETE /identity/v3/users/b4eae5b010d944abbc29fe2bc1ef2088 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:30:11.754347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00685a45-0a00-4719-80c6-2b76944cf830 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:11.756722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-00685a45-0a00-4719-80c6-2b76944cf830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:11.784592 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00685a45-0a00-4719-80c6-2b76944cf830 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:11.887112 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05638783-b902-44f7-a48a-9492bd837c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:11.887967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05638783-b902-44f7-a48a-9492bd837c24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:11.888255 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05638783-b902-44f7-a48a-9492bd837c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:11.888533 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05638783-b902-44f7-a48a-9492bd837c24 admin admin] PATH_INFO: `/v3/projects/a5dd92b2b227441b80d7cf164b574891` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:11.889794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05638783-b902-44f7-a48a-9492bd837c24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5dd92b2b227441b80d7cf164b574891)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:11.891747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05638783-b902-44f7-a48a-9492bd837c24 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:11.893180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:11.946706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-05638783-b902-44f7-a48a-9492bd837c24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5dd92b2b227441b80d7cf164b574891', 'request_id': 'req-05638783-b902-44f7-a48a-9492bd837c24'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:11.956421 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-05638783-b902-44f7-a48a-9492bd837c24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5dd92b2b227441b80d7cf164b574891', 'request_id': 'req-05638783-b902-44f7-a48a-9492bd837c24'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:11.964594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-05638783-b902-44f7-a48a-9492bd837c24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5dd92b2b227441b80d7cf164b574891', 'request_id': 'req-05638783-b902-44f7-a48a-9492bd837c24'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:11.986298 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-05638783-b902-44f7-a48a-9492bd837c24 admin admin] The token cache is being invalidate because project a5dd92b2b227441b80d7cf164b574891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:30:11.986593 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-05638783-b902-44f7-a48a-9492bd837c24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:11.987219 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-05638783-b902-44f7-a48a-9492bd837c24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:11.988766 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1072/2163] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:30:11 2022] DELETE /identity/v3/projects/a5dd92b2b227441b80d7cf164b574891 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:11.993773 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588076d-8361-47d0-aae8-8e0d9d8a261d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:11.994073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588076d-8361-47d0-aae8-8e0d9d8a261d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:11.994362 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588076d-8361-47d0-aae8-8e0d9d8a261d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:12.017094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2588076d-8361-47d0-aae8-8e0d9d8a261d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:12.018867 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2588076d-8361-47d0-aae8-8e0d9d8a261d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:12.087127 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1092/2164] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:30:12.095284 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cb7fe-28e7-4ad1-a15c-5920255371ba None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:12.095577 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cb7fe-28e7-4ad1-a15c-5920255371ba None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:12.095858 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cb7fe-28e7-4ad1-a15c-5920255371ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:12.114340 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ef7cb7fe-28e7-4ad1-a15c-5920255371ba None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:12.115748 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ef7cb7fe-28e7-4ad1-a15c-5920255371ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:12.124048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1073/2165] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:12.127816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2588076d-8361-47d0-aae8-8e0d9d8a261d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:12.137969 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2588076d-8361-47d0-aae8-8e0d9d8a261d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:12.143306 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e83bb-d047-484a-a2f9-c03f9f5aae5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:12.144113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e83bb-d047-484a-a2f9-c03f9f5aae5d admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:12.144402 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e83bb-d047-484a-a2f9-c03f9f5aae5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:12.144683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e83bb-d047-484a-a2f9-c03f9f5aae5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:12.145416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e83bb-d047-484a-a2f9-c03f9f5aae5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:30:12.145768 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e83bb-d047-484a-a2f9-c03f9f5aae5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:12.147422 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e83bb-d047-484a-a2f9-c03f9f5aae5d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:12.163378 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1093/2166] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:30:12.164581 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7cb7fe-28e7-4ad1-a15c-5920255371ba None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:12.174021 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7cb7fe-28e7-4ad1-a15c-5920255371ba None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:12.178788 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdbb6d9-3e80-484f-abe3-6f98af180da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:12.179433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbb6d9-3e80-484f-abe3-6f98af180da8 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:12.179717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbb6d9-3e80-484f-abe3-6f98af180da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:12.180000 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbb6d9-3e80-484f-abe3-6f98af180da8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:12.180745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdbb6d9-3e80-484f-abe3-6f98af180da8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:12.182699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdbb6d9-3e80-484f-abe3-6f98af180da8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:12.214887 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1074/2167] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:30:12 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:30:12.216052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e83bb-d047-484a-a2f9-c03f9f5aae5d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:12.226719 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e83bb-d047-484a-a2f9-c03f9f5aae5d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:12.305068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e312b4b-1019-471f-b543-f5e16130589b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:12.305774 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e312b4b-1019-471f-b543-f5e16130589b admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:12.306065 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e312b4b-1019-471f-b543-f5e16130589b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:12.306350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e312b4b-1019-471f-b543-f5e16130589b admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:12.307073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e312b4b-1019-471f-b543-f5e16130589b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:12.309006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e312b4b-1019-471f-b543-f5e16130589b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:12.340898 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1094/2168] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:30:12 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:30:12.342119 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdbb6d9-3e80-484f-abe3-6f98af180da8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:12.352626 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdbb6d9-3e80-484f-abe3-6f98af180da8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:12.357406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a468f-815a-43bd-989a-717b69c9e2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:12.358586 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a468f-815a-43bd-989a-717b69c9e2b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:12.358943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a468f-815a-43bd-989a-717b69c9e2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:12.359266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a468f-815a-43bd-989a-717b69c9e2b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:12.360003 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a468f-815a-43bd-989a-717b69c9e2b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:12.361652 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a468f-815a-43bd-989a-717b69c9e2b0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:12.375975 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1075/2169] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:12.377918 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e312b4b-1019-471f-b543-f5e16130589b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:12.390172 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e312b4b-1019-471f-b543-f5e16130589b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:12.397301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728d053-0775-4c18-93c1-d1d2698152c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:12.398575 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728d053-0775-4c18-93c1-d1d2698152c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:12.398969 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728d053-0775-4c18-93c1-d1d2698152c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:12.399354 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728d053-0775-4c18-93c1-d1d2698152c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:12.400321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728d053-0775-4c18-93c1-d1d2698152c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:12.403219 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728d053-0775-4c18-93c1-d1d2698152c9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:12.414583 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1095/2170] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:12.418747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a468f-815a-43bd-989a-717b69c9e2b0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:12.429778 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a468f-815a-43bd-989a-717b69c9e2b0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:12.434363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115ada2-e557-4f08-85a2-52abd3570da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:12.435226 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115ada2-e557-4f08-85a2-52abd3570da5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:12.435512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115ada2-e557-4f08-85a2-52abd3570da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:12.436309 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115ada2-e557-4f08-85a2-52abd3570da5 admin admin] PATH_INFO: `/v3/projects/6f04a401aa55491981e9b8d21c45ae71/users/02c0cdf0b82c4d83b2e17a2ab407fa79/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:12.452561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3115ada2-e557-4f08-85a2-52abd3570da5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f04a401aa55491981e9b8d21c45ae71, user_id=02c0cdf0b82c4d83b2e17a2ab407fa79, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:12.455376 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3115ada2-e557-4f08-85a2-52abd3570da5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:12.469477 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1076/2171] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:30:12 2022] PUT /identity/v3/projects/6f04a401aa55491981e9b8d21c45ae71/users/02c0cdf0b82c4d83b2e17a2ab407fa79/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:12.480621 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015191b-fdd4-4382-ad11-244f7f420bf4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:12.481191 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015191b-fdd4-4382-ad11-244f7f420bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:12.481671 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015191b-fdd4-4382-ad11-244f7f420bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:12.520395 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c015191b-fdd4-4382-ad11-244f7f420bf4 None None] MFA Rules not processed for user `02c0cdf0b82c4d83b2e17a2ab407fa79`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:12.522011 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c015191b-fdd4-4382-ad11-244f7f420bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:12.570909 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:12.604840 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1096/2172] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:12.610579 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115ada2-e557-4f08-85a2-52abd3570da5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:12.686197 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115ada2-e557-4f08-85a2-52abd3570da5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:12.706456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948796a-e904-46b1-83f4-2762963cce53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:12.707134 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948796a-e904-46b1-83f4-2762963cce53 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:12.707426 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948796a-e904-46b1-83f4-2762963cce53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:12.707710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948796a-e904-46b1-83f4-2762963cce53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:12.708411 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e948796a-e904-46b1-83f4-2762963cce53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:12.712299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e948796a-e904-46b1-83f4-2762963cce53 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:12.745712 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1077/2173] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:30:12 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:30:12.750169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015191b-fdd4-4382-ad11-244f7f420bf4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:12.763531 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015191b-fdd4-4382-ad11-244f7f420bf4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:12.825990 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fc151-a505-4bd0-aec3-643845d5ebd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:12.827316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fc151-a505-4bd0-aec3-643845d5ebd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:12.827718 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fc151-a505-4bd0-aec3-643845d5ebd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:12.828103 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fc151-a505-4bd0-aec3-643845d5ebd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:12.829876 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4fc151-a505-4bd0-aec3-643845d5ebd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:12.830988 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4fc151-a505-4bd0-aec3-643845d5ebd7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:12.853683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1097/2174] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:30:12 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:30:12.856568 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948796a-e904-46b1-83f4-2762963cce53 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:12.878961 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948796a-e904-46b1-83f4-2762963cce53 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:12.888417 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febfafdc-9770-47b8-b924-2591b83496f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:12.890193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfafdc-9770-47b8-b924-2591b83496f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:12.890829 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfafdc-9770-47b8-b924-2591b83496f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:12.894408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfafdc-9770-47b8-b924-2591b83496f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:12.894408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febfafdc-9770-47b8-b924-2591b83496f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:12.898105 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febfafdc-9770-47b8-b924-2591b83496f9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:12.906805 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:12.926517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1078/2175] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:12.933473 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fc151-a505-4bd0-aec3-643845d5ebd7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:12.953816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fc151-a505-4bd0-aec3-643845d5ebd7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:12.960141 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813251b0-d4a1-4ebe-889e-9fe07035b5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:12.961136 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813251b0-d4a1-4ebe-889e-9fe07035b5cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:12.961496 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813251b0-d4a1-4ebe-889e-9fe07035b5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:12.961879 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813251b0-d4a1-4ebe-889e-9fe07035b5cb admin admin] PATH_INFO: `/v3/projects/2b75c54b26154efe8efef09463def635/users/da92a72706a544c3bfa488ac45c3ce3b/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:12.978420 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813251b0-d4a1-4ebe-889e-9fe07035b5cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b75c54b26154efe8efef09463def635, user_id=da92a72706a544c3bfa488ac45c3ce3b, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:12.983829 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813251b0-d4a1-4ebe-889e-9fe07035b5cb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:13.005616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1098/2176] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:30:12 2022] PUT /identity/v3/projects/2b75c54b26154efe8efef09463def635/users/da92a72706a544c3bfa488ac45c3ce3b/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:13.009556 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febfafdc-9770-47b8-b924-2591b83496f9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:13.021492 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febfafdc-9770-47b8-b924-2591b83496f9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:13.111811 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34178b14-ae09-4608-a1bf-269a6d121dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:13.112479 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34178b14-ae09-4608-a1bf-269a6d121dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:13.112765 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34178b14-ae09-4608-a1bf-269a6d121dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:13.113046 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34178b14-ae09-4608-a1bf-269a6d121dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:13.113810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34178b14-ae09-4608-a1bf-269a6d121dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:13.115421 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34178b14-ae09-4608-a1bf-269a6d121dc1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:13.125957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1079/2177] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:13.130380 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813251b0-d4a1-4ebe-889e-9fe07035b5cb admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:13.141301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813251b0-d4a1-4ebe-889e-9fe07035b5cb admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:13.146114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851aea2-7f17-4d32-ab46-a75b7dcf6af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:13.147998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851aea2-7f17-4d32-ab46-a75b7dcf6af1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:13.148321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851aea2-7f17-4d32-ab46-a75b7dcf6af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:13.148604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851aea2-7f17-4d32-ab46-a75b7dcf6af1 admin admin] PATH_INFO: `/v3/projects/2b75c54b26154efe8efef09463def635/users/da92a72706a544c3bfa488ac45c3ce3b/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:13.150757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851aea2-7f17-4d32-ab46-a75b7dcf6af1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b75c54b26154efe8efef09463def635, user_id=da92a72706a544c3bfa488ac45c3ce3b, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:13.153565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851aea2-7f17-4d32-ab46-a75b7dcf6af1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:13.164804 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1099/2178] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:30:13 2022] PUT /identity/v3/projects/2b75c54b26154efe8efef09463def635/users/da92a72706a544c3bfa488ac45c3ce3b/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:13.177106 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc05f85-a548-42ea-aebd-cb695b2deac9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:13.177341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc05f85-a548-42ea-aebd-cb695b2deac9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:13.177681 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc05f85-a548-42ea-aebd-cb695b2deac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:13.205774 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8fc05f85-a548-42ea-aebd-cb695b2deac9 None None] MFA Rules not processed for user `da92a72706a544c3bfa488ac45c3ce3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:13.206981 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8fc05f85-a548-42ea-aebd-cb695b2deac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:13.283557 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1080/2179] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:13 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:13.294401 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851aea2-7f17-4d32-ab46-a75b7dcf6af1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:13.297106 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d851aea2-7f17-4d32-ab46-a75b7dcf6af1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:13.363000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851aea2-7f17-4d32-ab46-a75b7dcf6af1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:13.367160 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb64bd-94cd-400d-b2fe-19f15be8a36c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:13.368230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb64bd-94cd-400d-b2fe-19f15be8a36c service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:13.368661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb64bd-94cd-400d-b2fe-19f15be8a36c service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:13.369104 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb64bd-94cd-400d-b2fe-19f15be8a36c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:13.423896 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:13.437640 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb64bd-94cd-400d-b2fe-19f15be8a36c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:13.439036 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb64bd-94cd-400d-b2fe-19f15be8a36c service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:13.448184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1100/2180] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:19.124863 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc05f85-a548-42ea-aebd-cb695b2deac9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:19.194688 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc05f85-a548-42ea-aebd-cb695b2deac9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:19.199633 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21aee5b-618a-42e6-92c6-a94ebdeacc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:19.200510 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21aee5b-618a-42e6-92c6-a94ebdeacc86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:19.200795 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21aee5b-618a-42e6-92c6-a94ebdeacc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:19.201074 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21aee5b-618a-42e6-92c6-a94ebdeacc86 admin admin] PATH_INFO: `/v3/users/02c0cdf0b82c4d83b2e17a2ab407fa79` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:19.202474 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21aee5b-618a-42e6-92c6-a94ebdeacc86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02c0cdf0b82c4d83b2e17a2ab407fa79)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:19.204452 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21aee5b-618a-42e6-92c6-a94ebdeacc86 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:19.228502 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:19.257763 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e21aee5b-618a-42e6-92c6-a94ebdeacc86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02c0cdf0b82c4d83b2e17a2ab407fa79'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:19.258494 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e21aee5b-618a-42e6-92c6-a94ebdeacc86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02c0cdf0b82c4d83b2e17a2ab407fa79'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:19.270379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e21aee5b-618a-42e6-92c6-a94ebdeacc86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02c0cdf0b82c4d83b2e17a2ab407fa79'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:19.277768 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e21aee5b-618a-42e6-92c6-a94ebdeacc86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02c0cdf0b82c4d83b2e17a2ab407fa79'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:19.278359 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e21aee5b-618a-42e6-92c6-a94ebdeacc86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02c0cdf0b82c4d83b2e17a2ab407fa79'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:19.299169 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1081/2181] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:30:19 2022] DELETE /identity/v3/users/02c0cdf0b82c4d83b2e17a2ab407fa79 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:19.301245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a44bc-3d39-4270-be7b-660dbdf9c27b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:19.301750 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a44bc-3d39-4270-be7b-660dbdf9c27b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:19.301832 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a44bc-3d39-4270-be7b-660dbdf9c27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:19.320085 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fc6a44bc-3d39-4270-be7b-660dbdf9c27b None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:19.321324 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fc6a44bc-3d39-4270-be7b-660dbdf9c27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:19.398324 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1101/2182] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:30:19.408546 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21aee5b-618a-42e6-92c6-a94ebdeacc86 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:19.410213 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e21aee5b-618a-42e6-92c6-a94ebdeacc86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:19.503683 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21aee5b-618a-42e6-92c6-a94ebdeacc86 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:19.512116 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225e2dc-c403-41f8-9ef0-7e139806cc56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:19.512868 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225e2dc-c403-41f8-9ef0-7e139806cc56 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:19.513276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225e2dc-c403-41f8-9ef0-7e139806cc56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:19.513644 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225e2dc-c403-41f8-9ef0-7e139806cc56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:19.545129 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3225e2dc-c403-41f8-9ef0-7e139806cc56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:19.547164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3225e2dc-c403-41f8-9ef0-7e139806cc56 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:19.559623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1082/2183] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:20.215156 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a44bc-3d39-4270-be7b-660dbdf9c27b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:20.216591 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fc6a44bc-3d39-4270-be7b-660dbdf9c27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:20.231451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a44bc-3d39-4270-be7b-660dbdf9c27b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:20.237642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15e6a5-008f-45ad-975c-56b46b7e698f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:20.239696 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15e6a5-008f-45ad-975c-56b46b7e698f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:20.239696 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15e6a5-008f-45ad-975c-56b46b7e698f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:20.239696 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15e6a5-008f-45ad-975c-56b46b7e698f admin admin] PATH_INFO: `/v3/projects/6f04a401aa55491981e9b8d21c45ae71` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:20.248903 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15e6a5-008f-45ad-975c-56b46b7e698f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f04a401aa55491981e9b8d21c45ae71)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:20.251926 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15e6a5-008f-45ad-975c-56b46b7e698f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:20.309050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fd15e6a5-008f-45ad-975c-56b46b7e698f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f04a401aa55491981e9b8d21c45ae71', 'request_id': 'req-fd15e6a5-008f-45ad-975c-56b46b7e698f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:20.315826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fd15e6a5-008f-45ad-975c-56b46b7e698f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f04a401aa55491981e9b8d21c45ae71', 'request_id': 'req-fd15e6a5-008f-45ad-975c-56b46b7e698f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:20.321618 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fd15e6a5-008f-45ad-975c-56b46b7e698f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f04a401aa55491981e9b8d21c45ae71', 'request_id': 'req-fd15e6a5-008f-45ad-975c-56b46b7e698f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:20.341316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fd15e6a5-008f-45ad-975c-56b46b7e698f admin admin] The token cache is being invalidate because project 6f04a401aa55491981e9b8d21c45ae71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:30:20.341648 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fd15e6a5-008f-45ad-975c-56b46b7e698f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:20.343407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-fd15e6a5-008f-45ad-975c-56b46b7e698f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:20.345762 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1102/2184] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:30:20 2022] DELETE /identity/v3/projects/6f04a401aa55491981e9b8d21c45ae71 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:20.349838 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225e2dc-c403-41f8-9ef0-7e139806cc56 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:20.360658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225e2dc-c403-41f8-9ef0-7e139806cc56 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:20.431619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad882c-a495-4026-ae54-f4bf870ee9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:20.432489 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad882c-a495-4026-ae54-f4bf870ee9db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:20.432795 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad882c-a495-4026-ae54-f4bf870ee9db admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:20.433140 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad882c-a495-4026-ae54-f4bf870ee9db admin admin] PATH_INFO: `/v3/users/da92a72706a544c3bfa488ac45c3ce3b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:20.434528 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ad882c-a495-4026-ae54-f4bf870ee9db admin admin] RBAC: Authorizing `identity:delete_user(user_id=da92a72706a544c3bfa488ac45c3ce3b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:20.436520 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ad882c-a495-4026-ae54-f4bf870ee9db admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:20.491179 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:20.493008 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e2ad882c-a495-4026-ae54-f4bf870ee9db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da92a72706a544c3bfa488ac45c3ce3b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:20.493796 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e2ad882c-a495-4026-ae54-f4bf870ee9db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da92a72706a544c3bfa488ac45c3ce3b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:20.520347 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e2ad882c-a495-4026-ae54-f4bf870ee9db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da92a72706a544c3bfa488ac45c3ce3b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:20.528595 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e2ad882c-a495-4026-ae54-f4bf870ee9db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da92a72706a544c3bfa488ac45c3ce3b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:20.529280 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e2ad882c-a495-4026-ae54-f4bf870ee9db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da92a72706a544c3bfa488ac45c3ce3b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:20.541900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1083/2185] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:30:20 2022] DELETE /identity/v3/users/da92a72706a544c3bfa488ac45c3ce3b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:20.846827 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15e6a5-008f-45ad-975c-56b46b7e698f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:20.854492 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fd15e6a5-008f-45ad-975c-56b46b7e698f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:20.885257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15e6a5-008f-45ad-975c-56b46b7e698f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:21.034635 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a7f1e-1cd4-4cce-8492-d7f1474f64f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:21.035772 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a7f1e-1cd4-4cce-8492-d7f1474f64f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:21.036131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a7f1e-1cd4-4cce-8492-d7f1474f64f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:21.036504 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a7f1e-1cd4-4cce-8492-d7f1474f64f4 admin admin] PATH_INFO: `/v3/projects/2b75c54b26154efe8efef09463def635` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:21.037910 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a7f1e-1cd4-4cce-8492-d7f1474f64f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b75c54b26154efe8efef09463def635)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:21.040142 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a7f1e-1cd4-4cce-8492-d7f1474f64f4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:21.053721 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:21.120340 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-896a7f1e-1cd4-4cce-8492-d7f1474f64f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b75c54b26154efe8efef09463def635', 'request_id': 'req-896a7f1e-1cd4-4cce-8492-d7f1474f64f4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:21.129780 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-896a7f1e-1cd4-4cce-8492-d7f1474f64f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b75c54b26154efe8efef09463def635', 'request_id': 'req-896a7f1e-1cd4-4cce-8492-d7f1474f64f4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:21.141795 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-896a7f1e-1cd4-4cce-8492-d7f1474f64f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b75c54b26154efe8efef09463def635', 'request_id': 'req-896a7f1e-1cd4-4cce-8492-d7f1474f64f4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:21.158768 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-896a7f1e-1cd4-4cce-8492-d7f1474f64f4 admin admin] The token cache is being invalidate because project 2b75c54b26154efe8efef09463def635 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:30:21.159135 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-896a7f1e-1cd4-4cce-8492-d7f1474f64f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:21.160046 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-896a7f1e-1cd4-4cce-8492-d7f1474f64f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:21.162012 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1103/2186] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:30:20 2022] DELETE /identity/v3/projects/2b75c54b26154efe8efef09463def635 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:21.166938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a265d-e5ba-49a5-9ea5-c130fcd64ccb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:21.167234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a265d-e5ba-49a5-9ea5-c130fcd64ccb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:21.167514 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a265d-e5ba-49a5-9ea5-c130fcd64ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:21.185934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b60a265d-e5ba-49a5-9ea5-c130fcd64ccb None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:21.187372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b60a265d-e5ba-49a5-9ea5-c130fcd64ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:21.254739 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1084/2187] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:21.263695 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369587f-ee68-470f-a62d-5a91f28b4fd8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:21.264165 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369587f-ee68-470f-a62d-5a91f28b4fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:21.264952 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369587f-ee68-470f-a62d-5a91f28b4fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:21.284894 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5369587f-ee68-470f-a62d-5a91f28b4fd8 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:21.286148 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5369587f-ee68-470f-a62d-5a91f28b4fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:21.294344 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1104/2188] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:21.298910 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a265d-e5ba-49a5-9ea5-c130fcd64ccb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:21.318063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a265d-e5ba-49a5-9ea5-c130fcd64ccb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:21.323029 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5bf12-5da2-45b1-bf32-423215d8eeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:21.323692 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5bf12-5da2-45b1-bf32-423215d8eeba admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:21.323980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5bf12-5da2-45b1-bf32-423215d8eeba admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:21.324255 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5bf12-5da2-45b1-bf32-423215d8eeba admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:21.325028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5bf12-5da2-45b1-bf32-423215d8eeba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:30:21.325339 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5bf12-5da2-45b1-bf32-423215d8eeba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:21.327679 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5bf12-5da2-45b1-bf32-423215d8eeba admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:21.340204 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1085/2189] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:30:21.344829 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369587f-ee68-470f-a62d-5a91f28b4fd8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:21.355293 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369587f-ee68-470f-a62d-5a91f28b4fd8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:21.360613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279d43f-92fe-4333-8685-ba5e80c1eb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:21.361342 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279d43f-92fe-4333-8685-ba5e80c1eb67 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:21.361739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279d43f-92fe-4333-8685-ba5e80c1eb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:21.362104 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279d43f-92fe-4333-8685-ba5e80c1eb67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:21.362859 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279d43f-92fe-4333-8685-ba5e80c1eb67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:21.364962 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279d43f-92fe-4333-8685-ba5e80c1eb67 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:21.394036 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1105/2190] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:30:21 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:30:21.398290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5bf12-5da2-45b1-bf32-423215d8eeba admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:21.425332 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5bf12-5da2-45b1-bf32-423215d8eeba admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:21.504526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd359c4-e0ad-4150-a28a-33ad76341a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:21.505240 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd359c4-e0ad-4150-a28a-33ad76341a52 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:21.505650 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd359c4-e0ad-4150-a28a-33ad76341a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:21.505934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd359c4-e0ad-4150-a28a-33ad76341a52 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:21.506773 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd359c4-e0ad-4150-a28a-33ad76341a52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:21.509020 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd359c4-e0ad-4150-a28a-33ad76341a52 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:21.543644 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1086/2191] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:30:21 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:30:21.547994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279d43f-92fe-4333-8685-ba5e80c1eb67 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:21.565430 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279d43f-92fe-4333-8685-ba5e80c1eb67 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:21.571403 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ef790-8455-4718-934f-807f86a3d173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:21.574108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ef790-8455-4718-934f-807f86a3d173 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:21.574515 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ef790-8455-4718-934f-807f86a3d173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:21.574904 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ef790-8455-4718-934f-807f86a3d173 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:21.575939 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ef790-8455-4718-934f-807f86a3d173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:21.578086 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ef790-8455-4718-934f-807f86a3d173 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:21.590537 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1106/2192] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:30:21.595420 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd359c4-e0ad-4150-a28a-33ad76341a52 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:21.621864 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd359c4-e0ad-4150-a28a-33ad76341a52 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:21.627685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec20cc4-05e6-458f-8e83-a44c4a68a589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:21.628550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec20cc4-05e6-458f-8e83-a44c4a68a589 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:21.628838 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec20cc4-05e6-458f-8e83-a44c4a68a589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:21.629151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec20cc4-05e6-458f-8e83-a44c4a68a589 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:21.629930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec20cc4-05e6-458f-8e83-a44c4a68a589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:21.631557 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec20cc4-05e6-458f-8e83-a44c4a68a589 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:21.648824 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1087/2193] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:21.652679 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ef790-8455-4718-934f-807f86a3d173 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:21.671482 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ef790-8455-4718-934f-807f86a3d173 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:21.676615 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7efb1a-163c-485c-81ed-f10497241b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:21.677686 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7efb1a-163c-485c-81ed-f10497241b41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:21.677978 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7efb1a-163c-485c-81ed-f10497241b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:21.678257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7efb1a-163c-485c-81ed-f10497241b41 admin admin] PATH_INFO: `/v3/projects/0ca03c0d189a402389e51c76437bae45/users/919b4079d92f4a80af4869df4584e46e/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:21.699599 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7efb1a-163c-485c-81ed-f10497241b41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ca03c0d189a402389e51c76437bae45, user_id=919b4079d92f4a80af4869df4584e46e, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:21.704890 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7efb1a-163c-485c-81ed-f10497241b41 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:21.724121 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1107/2194] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:30:21 2022] PUT /identity/v3/projects/0ca03c0d189a402389e51c76437bae45/users/919b4079d92f4a80af4869df4584e46e/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:21.735409 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e846f-e5e9-435f-b580-faf1cf718398 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:21.735778 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e846f-e5e9-435f-b580-faf1cf718398 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:21.736133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e846f-e5e9-435f-b580-faf1cf718398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:21.785992 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e55e846f-e5e9-435f-b580-faf1cf718398 None None] MFA Rules not processed for user `919b4079d92f4a80af4869df4584e46e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:21.787383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e55e846f-e5e9-435f-b580-faf1cf718398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:21.811918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:21.886861 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1088/2195] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:30:21.891610 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7efb1a-163c-485c-81ed-f10497241b41 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:22.084238 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7efb1a-163c-485c-81ed-f10497241b41 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:22.090336 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f837447-1d2c-4a0e-8998-c5dca064168c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:22.091245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f837447-1d2c-4a0e-8998-c5dca064168c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:22.091704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f837447-1d2c-4a0e-8998-c5dca064168c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:22.092133 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f837447-1d2c-4a0e-8998-c5dca064168c admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:22.098803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f837447-1d2c-4a0e-8998-c5dca064168c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:22.101262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f837447-1d2c-4a0e-8998-c5dca064168c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:22.152472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1108/2196] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:30:21 2022] POST /identity/v3/projects => generated 383 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:30:22.156754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e846f-e5e9-435f-b580-faf1cf718398 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:22.179447 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e846f-e5e9-435f-b580-faf1cf718398 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:22.296314 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd968c6c-4c58-4bbc-b194-dd00fbc3e521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:22.297092 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd968c6c-4c58-4bbc-b194-dd00fbc3e521 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:22.297410 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd968c6c-4c58-4bbc-b194-dd00fbc3e521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:22.298046 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd968c6c-4c58-4bbc-b194-dd00fbc3e521 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:22.299027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd968c6c-4c58-4bbc-b194-dd00fbc3e521 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:22.301216 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd968c6c-4c58-4bbc-b194-dd00fbc3e521 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:22.327734 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1089/2197] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:30:22 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:30:22.332666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f837447-1d2c-4a0e-8998-c5dca064168c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:22.343336 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f837447-1d2c-4a0e-8998-c5dca064168c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:22.348542 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3ed16-de30-4b5f-9d53-3abc7d581d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:22.349225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3ed16-de30-4b5f-9d53-3abc7d581d81 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:22.349514 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3ed16-de30-4b5f-9d53-3abc7d581d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:22.350561 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3ed16-de30-4b5f-9d53-3abc7d581d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:22.351315 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d3ed16-de30-4b5f-9d53-3abc7d581d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:22.352951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d3ed16-de30-4b5f-9d53-3abc7d581d81 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:22.366396 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1109/2198] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:22.370179 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd968c6c-4c58-4bbc-b194-dd00fbc3e521 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:22.380956 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd968c6c-4c58-4bbc-b194-dd00fbc3e521 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:22.387195 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9511917-7c58-4538-ba75-5d0debc7f679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:22.388021 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9511917-7c58-4538-ba75-5d0debc7f679 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:22.388379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9511917-7c58-4538-ba75-5d0debc7f679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:22.388728 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9511917-7c58-4538-ba75-5d0debc7f679 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:22.389617 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9511917-7c58-4538-ba75-5d0debc7f679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:22.391326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9511917-7c58-4538-ba75-5d0debc7f679 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:22.405630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1090/2199] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:22.407373 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3ed16-de30-4b5f-9d53-3abc7d581d81 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:22.422060 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3ed16-de30-4b5f-9d53-3abc7d581d81 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:22.427221 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a329e-6cd6-4b07-be0c-65ea5e362744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:22.428316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a329e-6cd6-4b07-be0c-65ea5e362744 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:22.428638 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a329e-6cd6-4b07-be0c-65ea5e362744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:22.428986 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a329e-6cd6-4b07-be0c-65ea5e362744 admin admin] PATH_INFO: `/v3/projects/06668268a675484aa22e45a49de74291/users/ef97ce69f08d4c6b926128de7e936bc9/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:22.432814 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:22.447849 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a329e-6cd6-4b07-be0c-65ea5e362744 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06668268a675484aa22e45a49de74291, user_id=ef97ce69f08d4c6b926128de7e936bc9, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:22.450610 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a329e-6cd6-4b07-be0c-65ea5e362744 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:22.464353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1110/2200] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:30:22 2022] PUT /identity/v3/projects/06668268a675484aa22e45a49de74291/users/ef97ce69f08d4c6b926128de7e936bc9/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:22.474565 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a81f9-c26e-4b11-8ceb-154a31e609df None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:22.474854 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a81f9-c26e-4b11-8ceb-154a31e609df None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:22.475132 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a81f9-c26e-4b11-8ceb-154a31e609df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:22.503495 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2e6a81f9-c26e-4b11-8ceb-154a31e609df None None] MFA Rules not processed for user `ef97ce69f08d4c6b926128de7e936bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:22.504716 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2e6a81f9-c26e-4b11-8ceb-154a31e609df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:22.568898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1091/2201] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:22.573428 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a329e-6cd6-4b07-be0c-65ea5e362744 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:22.640309 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a329e-6cd6-4b07-be0c-65ea5e362744 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:22.645050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bdf6f-2d9a-4e4f-b6c8-c4d5e503505c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:22.645821 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bdf6f-2d9a-4e4f-b6c8-c4d5e503505c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:22.646114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bdf6f-2d9a-4e4f-b6c8-c4d5e503505c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:22.646395 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bdf6f-2d9a-4e4f-b6c8-c4d5e503505c admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:22.647093 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bdf6f-2d9a-4e4f-b6c8-c4d5e503505c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:22.649016 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bdf6f-2d9a-4e4f-b6c8-c4d5e503505c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:22.673985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1111/2202] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:30:22 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:30:22.678168 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a81f9-c26e-4b11-8ceb-154a31e609df None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:22.689480 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a81f9-c26e-4b11-8ceb-154a31e609df None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:22.716418 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:22.751846 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa42f90-9e94-4b88-8c44-e37ed063efdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:22.752514 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa42f90-9e94-4b88-8c44-e37ed063efdd admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:22.752807 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa42f90-9e94-4b88-8c44-e37ed063efdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:22.753087 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa42f90-9e94-4b88-8c44-e37ed063efdd admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:22.753824 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa42f90-9e94-4b88-8c44-e37ed063efdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:22.755757 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa42f90-9e94-4b88-8c44-e37ed063efdd admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:22.777056 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1092/2203] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:30:22 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:30:22.781533 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bdf6f-2d9a-4e4f-b6c8-c4d5e503505c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:22.791927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bdf6f-2d9a-4e4f-b6c8-c4d5e503505c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:22.797910 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2d1a5-895f-479d-801e-32d3b0c843a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:22.798655 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2d1a5-895f-479d-801e-32d3b0c843a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:22.799010 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2d1a5-895f-479d-801e-32d3b0c843a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:22.799407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2d1a5-895f-479d-801e-32d3b0c843a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:22.800293 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b2d1a5-895f-479d-801e-32d3b0c843a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:22.802054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b2d1a5-895f-479d-801e-32d3b0c843a8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:22.813448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1112/2204] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:30:22.819790 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa42f90-9e94-4b88-8c44-e37ed063efdd admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:22.830451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa42f90-9e94-4b88-8c44-e37ed063efdd admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:22.835106 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6334720-89e4-4188-b7ab-0056843a5898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:22.835990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6334720-89e4-4188-b7ab-0056843a5898 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:22.836271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6334720-89e4-4188-b7ab-0056843a5898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:22.836548 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6334720-89e4-4188-b7ab-0056843a5898 admin admin] PATH_INFO: `/v3/projects/d582c012fc614a9f87a0d470d6467375/users/c811254ec65843b9af78f8c64f621536/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:22.852315 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6334720-89e4-4188-b7ab-0056843a5898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d582c012fc614a9f87a0d470d6467375, user_id=c811254ec65843b9af78f8c64f621536, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:22.855025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6334720-89e4-4188-b7ab-0056843a5898 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:22.869011 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1093/2205] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:30:22 2022] PUT /identity/v3/projects/d582c012fc614a9f87a0d470d6467375/users/c811254ec65843b9af78f8c64f621536/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:22.873107 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2d1a5-895f-479d-801e-32d3b0c843a8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:22.883508 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2d1a5-895f-479d-801e-32d3b0c843a8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:22.944224 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3517c279-61ba-41ef-b1c0-124265271380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:22.944892 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517c279-61ba-41ef-b1c0-124265271380 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:22.945178 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517c279-61ba-41ef-b1c0-124265271380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:22.945455 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517c279-61ba-41ef-b1c0-124265271380 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:22.946216 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517c279-61ba-41ef-b1c0-124265271380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:22.947817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517c279-61ba-41ef-b1c0-124265271380 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:22.957663 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1113/2206] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:22.961785 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6334720-89e4-4188-b7ab-0056843a5898 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:22.972640 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6334720-89e4-4188-b7ab-0056843a5898 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:22.977246 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c3f4d-0ba0-4d36-8bcd-a85a565e2536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:22.978141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c3f4d-0ba0-4d36-8bcd-a85a565e2536 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:22.978427 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c3f4d-0ba0-4d36-8bcd-a85a565e2536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:22.978705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c3f4d-0ba0-4d36-8bcd-a85a565e2536 admin admin] PATH_INFO: `/v3/projects/d582c012fc614a9f87a0d470d6467375/users/c811254ec65843b9af78f8c64f621536/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:22.980792 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4c3f4d-0ba0-4d36-8bcd-a85a565e2536 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d582c012fc614a9f87a0d470d6467375, user_id=c811254ec65843b9af78f8c64f621536, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:22.984138 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4c3f4d-0ba0-4d36-8bcd-a85a565e2536 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:22.994404 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1094/2207] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:30:22 2022] PUT /identity/v3/projects/d582c012fc614a9f87a0d470d6467375/users/c811254ec65843b9af78f8c64f621536/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:23.062681 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cfdf1-ab92-40db-b340-c8848c162767 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:23.062973 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cfdf1-ab92-40db-b340-c8848c162767 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:23.063252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cfdf1-ab92-40db-b340-c8848c162767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:23.095206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c73cfdf1-ab92-40db-b340-c8848c162767 None None] MFA Rules not processed for user `c811254ec65843b9af78f8c64f621536`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:23.096669 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c73cfdf1-ab92-40db-b340-c8848c162767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:23.172333 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:23.199872 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1114/2208] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:23.211507 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c3f4d-0ba0-4d36-8bcd-a85a565e2536 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:23.213487 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5c4c3f4d-0ba0-4d36-8bcd-a85a565e2536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:23.279455 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c3f4d-0ba0-4d36-8bcd-a85a565e2536 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:23.283491 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed80eb-f3f8-411b-a87d-6a28de2f5216 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:23.284134 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed80eb-f3f8-411b-a87d-6a28de2f5216 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:23.284413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed80eb-f3f8-411b-a87d-6a28de2f5216 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:23.284689 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed80eb-f3f8-411b-a87d-6a28de2f5216 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:23.348535 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed80eb-f3f8-411b-a87d-6a28de2f5216 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:23.349954 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed80eb-f3f8-411b-a87d-6a28de2f5216 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:23.362640 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1095/2209] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:29.176964 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73cfdf1-ab92-40db-b340-c8848c162767 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:29.190149 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73cfdf1-ab92-40db-b340-c8848c162767 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:29.195470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546cac62-db8c-4bc5-9ec2-2b8e8e5e8eee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:29.195613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546cac62-db8c-4bc5-9ec2-2b8e8e5e8eee service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:29.196004 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546cac62-db8c-4bc5-9ec2-2b8e8e5e8eee service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:29.196202 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546cac62-db8c-4bc5-9ec2-2b8e8e5e8eee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:29.282887 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546cac62-db8c-4bc5-9ec2-2b8e8e5e8eee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:29.284408 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546cac62-db8c-4bc5-9ec2-2b8e8e5e8eee service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:29.297123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1115/2210] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:30.020814 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed80eb-f3f8-411b-a87d-6a28de2f5216 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:30.027775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1aed80eb-f3f8-411b-a87d-6a28de2f5216 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:30.040117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed80eb-f3f8-411b-a87d-6a28de2f5216 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:30.043731 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f00457-13c5-4640-9258-a4a1e517fbac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:30.044411 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f00457-13c5-4640-9258-a4a1e517fbac service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:30.044698 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f00457-13c5-4640-9258-a4a1e517fbac service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:30.044979 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f00457-13c5-4640-9258-a4a1e517fbac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:30.056689 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f00457-13c5-4640-9258-a4a1e517fbac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:30.058127 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f00457-13c5-4640-9258-a4a1e517fbac service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:30.073547 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1096/2211] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:31.696596 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546cac62-db8c-4bc5-9ec2-2b8e8e5e8eee service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:31.699000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-546cac62-db8c-4bc5-9ec2-2b8e8e5e8eee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:31.776086 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546cac62-db8c-4bc5-9ec2-2b8e8e5e8eee service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:31.780778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e84f3-4ed5-466a-8452-dc144c79233f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:31.781678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e84f3-4ed5-466a-8452-dc144c79233f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:31.781989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e84f3-4ed5-466a-8452-dc144c79233f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:31.782270 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e84f3-4ed5-466a-8452-dc144c79233f admin admin] PATH_INFO: `/v3/users/6aeea7135604416e962f599e64424173` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:31.783464 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e84f3-4ed5-466a-8452-dc144c79233f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aeea7135604416e962f599e64424173)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:31.785448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e84f3-4ed5-466a-8452-dc144c79233f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:31.833790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6a0e84f3-4ed5-466a-8452-dc144c79233f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aeea7135604416e962f599e64424173'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:31.846888 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6a0e84f3-4ed5-466a-8452-dc144c79233f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aeea7135604416e962f599e64424173'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:31.847660 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6a0e84f3-4ed5-466a-8452-dc144c79233f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aeea7135604416e962f599e64424173'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:31.855703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6a0e84f3-4ed5-466a-8452-dc144c79233f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aeea7135604416e962f599e64424173'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:31.856346 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6a0e84f3-4ed5-466a-8452-dc144c79233f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aeea7135604416e962f599e64424173'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:31.866207 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:31.872516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1116/2212] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:30:31 2022] DELETE /identity/v3/users/6aeea7135604416e962f599e64424173 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:31.877150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44737dc9-11e8-4c78-81f2-49a043b66c3d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:31.877685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44737dc9-11e8-4c78-81f2-49a043b66c3d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:31.877811 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44737dc9-11e8-4c78-81f2-49a043b66c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:31.895203 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-44737dc9-11e8-4c78-81f2-49a043b66c3d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:31.896394 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-44737dc9-11e8-4c78-81f2-49a043b66c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:31.910408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:31.972256 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1097/2213] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:31.981992 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e84f3-4ed5-466a-8452-dc144c79233f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:31.983478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6a0e84f3-4ed5-466a-8452-dc144c79233f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:32.056619 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e84f3-4ed5-466a-8452-dc144c79233f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:32.060590 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381ab57-6d06-430b-881c-db68d887c666 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:32.062170 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381ab57-6d06-430b-881c-db68d887c666 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:32.062671 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381ab57-6d06-430b-881c-db68d887c666 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:32.062961 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381ab57-6d06-430b-881c-db68d887c666 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:32.073108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a381ab57-6d06-430b-881c-db68d887c666 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:32.074619 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a381ab57-6d06-430b-881c-db68d887c666 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:32.085407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1117/2214] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:32.391767 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44737dc9-11e8-4c78-81f2-49a043b66c3d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:32.393514 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-44737dc9-11e8-4c78-81f2-49a043b66c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:32.410083 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44737dc9-11e8-4c78-81f2-49a043b66c3d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:32.416375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45696f26-12c3-42af-b7f3-84a3858aebf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:32.417254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45696f26-12c3-42af-b7f3-84a3858aebf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:32.417537 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45696f26-12c3-42af-b7f3-84a3858aebf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:32.417847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45696f26-12c3-42af-b7f3-84a3858aebf3 admin admin] PATH_INFO: `/v3/projects/02241431a7964f609e505e6618133e46` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:32.419907 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45696f26-12c3-42af-b7f3-84a3858aebf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02241431a7964f609e505e6618133e46)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:32.422014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45696f26-12c3-42af-b7f3-84a3858aebf3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:32.475764 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-45696f26-12c3-42af-b7f3-84a3858aebf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02241431a7964f609e505e6618133e46', 'request_id': 'req-45696f26-12c3-42af-b7f3-84a3858aebf3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:32.483775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-45696f26-12c3-42af-b7f3-84a3858aebf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02241431a7964f609e505e6618133e46', 'request_id': 'req-45696f26-12c3-42af-b7f3-84a3858aebf3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:32.490083 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-45696f26-12c3-42af-b7f3-84a3858aebf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02241431a7964f609e505e6618133e46', 'request_id': 'req-45696f26-12c3-42af-b7f3-84a3858aebf3'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:32.505943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-45696f26-12c3-42af-b7f3-84a3858aebf3 admin admin] The token cache is being invalidate because project 02241431a7964f609e505e6618133e46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:30:32.506244 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-45696f26-12c3-42af-b7f3-84a3858aebf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:32.506880 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-45696f26-12c3-42af-b7f3-84a3858aebf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:32.508624 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1098/2215] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:30:32 2022] DELETE /identity/v3/projects/02241431a7964f609e505e6618133e46 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:30:32.513679 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20647c1-66e1-4f6e-95dc-ecea9b68b930 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:32.514012 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20647c1-66e1-4f6e-95dc-ecea9b68b930 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:32.514511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20647c1-66e1-4f6e-95dc-ecea9b68b930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:32.535577 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c20647c1-66e1-4f6e-95dc-ecea9b68b930 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:32.537045 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c20647c1-66e1-4f6e-95dc-ecea9b68b930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:32.629965 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1118/2216] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:32.635479 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997dbb1-5155-4a7a-908f-946a441f353a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:32.635789 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997dbb1-5155-4a7a-908f-946a441f353a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:32.636047 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997dbb1-5155-4a7a-908f-946a441f353a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:32.664047 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-d997dbb1-5155-4a7a-908f-946a441f353a None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:32.666111 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d997dbb1-5155-4a7a-908f-946a441f353a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:32.677679 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1099/2217] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:32.680700 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20647c1-66e1-4f6e-95dc-ecea9b68b930 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:32.697992 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20647c1-66e1-4f6e-95dc-ecea9b68b930 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:32.711339 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f88d8-c906-44ab-ace8-f097b71c142d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:32.712243 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f88d8-c906-44ab-ace8-f097b71c142d admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:32.712900 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f88d8-c906-44ab-ace8-f097b71c142d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:32.713349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f88d8-c906-44ab-ace8-f097b71c142d admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:32.714297 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f88d8-c906-44ab-ace8-f097b71c142d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:30:32.715410 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f88d8-c906-44ab-ace8-f097b71c142d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:32.717062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f88d8-c906-44ab-ace8-f097b71c142d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:32.731817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1119/2218] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:30:32.738469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997dbb1-5155-4a7a-908f-946a441f353a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:32.751541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997dbb1-5155-4a7a-908f-946a441f353a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:32.775477 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f278f3d-f092-403f-b257-9724b8d49c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:32.776171 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f278f3d-f092-403f-b257-9724b8d49c47 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:32.776457 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f278f3d-f092-403f-b257-9724b8d49c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:32.776736 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f278f3d-f092-403f-b257-9724b8d49c47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:32.777455 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f278f3d-f092-403f-b257-9724b8d49c47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:32.781315 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f278f3d-f092-403f-b257-9724b8d49c47 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:32.835547 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1100/2219] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:30:32 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:30:32.839932 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f88d8-c906-44ab-ace8-f097b71c142d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:32.851893 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f88d8-c906-44ab-ace8-f097b71c142d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:32.985524 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2ddb5-b767-4ba3-ad60-90ad75cd1f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:32.986353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2ddb5-b767-4ba3-ad60-90ad75cd1f30 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:32.986709 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2ddb5-b767-4ba3-ad60-90ad75cd1f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:32.987061 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2ddb5-b767-4ba3-ad60-90ad75cd1f30 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:32.988839 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e2ddb5-b767-4ba3-ad60-90ad75cd1f30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:32.991093 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e2ddb5-b767-4ba3-ad60-90ad75cd1f30 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:33.040148 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1120/2220] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:30:32 2022] POST /identity/v3/users => generated 287 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:30:33.042578 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f278f3d-f092-403f-b257-9724b8d49c47 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:33.054796 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f278f3d-f092-403f-b257-9724b8d49c47 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:33.071142 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23fb183-ce8c-4289-bc99-f4ee033877e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:33.071729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fb183-ce8c-4289-bc99-f4ee033877e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:33.072212 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fb183-ce8c-4289-bc99-f4ee033877e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:33.072324 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fb183-ce8c-4289-bc99-f4ee033877e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:33.073009 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23fb183-ce8c-4289-bc99-f4ee033877e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:33.074714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23fb183-ce8c-4289-bc99-f4ee033877e8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:33.086982 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1101/2221] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:33.094676 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2ddb5-b767-4ba3-ad60-90ad75cd1f30 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:33.108649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2ddb5-b767-4ba3-ad60-90ad75cd1f30 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:33.114324 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b79c5-a349-47c3-b244-1bf4435f1109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:33.115080 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b79c5-a349-47c3-b244-1bf4435f1109 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:33.115367 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b79c5-a349-47c3-b244-1bf4435f1109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:33.116021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b79c5-a349-47c3-b244-1bf4435f1109 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:33.116826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b79c5-a349-47c3-b244-1bf4435f1109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:33.118664 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b79c5-a349-47c3-b244-1bf4435f1109 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:33.133437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1121/2222] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:30:33.138354 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23fb183-ce8c-4289-bc99-f4ee033877e8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:33.151140 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23fb183-ce8c-4289-bc99-f4ee033877e8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:33.157832 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf27bdb-4e1b-401f-b1dd-178a44618ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:33.158758 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf27bdb-4e1b-401f-b1dd-178a44618ead admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:33.159045 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf27bdb-4e1b-401f-b1dd-178a44618ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:33.159322 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf27bdb-4e1b-401f-b1dd-178a44618ead admin admin] PATH_INFO: `/v3/projects/116a07f168ef45e1b2f4e7a084fba32c/users/374555cda1b4424894093ebcc51cd651/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:33.183720 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf27bdb-4e1b-401f-b1dd-178a44618ead admin admin] RBAC: Authorizing `identity:create_grant(project_id=116a07f168ef45e1b2f4e7a084fba32c, user_id=374555cda1b4424894093ebcc51cd651, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:33.187971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf27bdb-4e1b-401f-b1dd-178a44618ead admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:33.208332 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1102/2223] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:30:33 2022] PUT /identity/v3/projects/116a07f168ef45e1b2f4e7a084fba32c/users/374555cda1b4424894093ebcc51cd651/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:33.212898 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489f0b1-bad6-41ab-84fb-43c5751dae66 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:33.213193 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489f0b1-bad6-41ab-84fb-43c5751dae66 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:33.213475 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489f0b1-bad6-41ab-84fb-43c5751dae66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:33.233492 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:33.241424 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1489f0b1-bad6-41ab-84fb-43c5751dae66 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:33.242795 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1489f0b1-bad6-41ab-84fb-43c5751dae66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:33.306582 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1122/2224] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:30:33.316567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf27bdb-4e1b-401f-b1dd-178a44618ead admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:33.318451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bdf27bdb-4e1b-401f-b1dd-178a44618ead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:33.364357 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:33.380095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf27bdb-4e1b-401f-b1dd-178a44618ead admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:33.384017 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1c21d-f2c5-4c74-9dae-538a115cdf43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:33.384668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1c21d-f2c5-4c74-9dae-538a115cdf43 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:33.384958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1c21d-f2c5-4c74-9dae-538a115cdf43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:33.385241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1c21d-f2c5-4c74-9dae-538a115cdf43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:33.395000 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1c21d-f2c5-4c74-9dae-538a115cdf43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:33.396530 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1c21d-f2c5-4c74-9dae-538a115cdf43 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:33.410315 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1103/2225] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:34.310533 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2227ea2-a2b4-48eb-a5d2-e4e18f07de5e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:34.312121 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2227ea2-a2b4-48eb-a5d2-e4e18f07de5e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:34.312651 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2227ea2-a2b4-48eb-a5d2-e4e18f07de5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:34.350543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e2227ea2-a2b4-48eb-a5d2-e4e18f07de5e None None] MFA Rules not processed for user `374555cda1b4424894093ebcc51cd651`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:34.350698 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1c21d-f2c5-4c74-9dae-538a115cdf43 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:34.352126 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-65c1c21d-f2c5-4c74-9dae-538a115cdf43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:34.352327 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e2227ea2-a2b4-48eb-a5d2-e4e18f07de5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:34.374337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1c21d-f2c5-4c74-9dae-538a115cdf43 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:34.384363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e952c70-1751-4ffb-a0a6-f2f5e7174268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:34.385525 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e952c70-1751-4ffb-a0a6-f2f5e7174268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:34.386172 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e952c70-1751-4ffb-a0a6-f2f5e7174268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:34.387355 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e952c70-1751-4ffb-a0a6-f2f5e7174268 admin admin] PATH_INFO: `/v3/users/919b4079d92f4a80af4869df4584e46e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:34.389484 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e952c70-1751-4ffb-a0a6-f2f5e7174268 admin admin] RBAC: Authorizing `identity:delete_user(user_id=919b4079d92f4a80af4869df4584e46e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:34.395492 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e952c70-1751-4ffb-a0a6-f2f5e7174268 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:34.460664 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1123/2226] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:34.465228 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9e952c70-1751-4ffb-a0a6-f2f5e7174268 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '919b4079d92f4a80af4869df4584e46e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:34.466247 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9e952c70-1751-4ffb-a0a6-f2f5e7174268 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '919b4079d92f4a80af4869df4584e46e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:34.475692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2227ea2-a2b4-48eb-a5d2-e4e18f07de5e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:34.477224 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e2227ea2-a2b4-48eb-a5d2-e4e18f07de5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:34.481646 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9e952c70-1751-4ffb-a0a6-f2f5e7174268 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '919b4079d92f4a80af4869df4584e46e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:34.496649 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9e952c70-1751-4ffb-a0a6-f2f5e7174268 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '919b4079d92f4a80af4869df4584e46e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:34.498240 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9e952c70-1751-4ffb-a0a6-f2f5e7174268 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '919b4079d92f4a80af4869df4584e46e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:34.529407 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1104/2227] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:30:34 2022] DELETE /identity/v3/users/919b4079d92f4a80af4869df4584e46e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:34.537512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e0d05-9690-40a1-830f-02cb5d579cf7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:34.537915 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e0d05-9690-40a1-830f-02cb5d579cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:34.538230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e0d05-9690-40a1-830f-02cb5d579cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:34.558881 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2227ea2-a2b4-48eb-a5d2-e4e18f07de5e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:34.566837 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8d5e0d05-9690-40a1-830f-02cb5d579cf7 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:34.568209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8d5e0d05-9690-40a1-830f-02cb5d579cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:34.633291 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3ad80-50fd-4da7-ab6c-d8a95fbe73ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:34.634126 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3ad80-50fd-4da7-ab6c-d8a95fbe73ff service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:34.634547 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3ad80-50fd-4da7-ab6c-d8a95fbe73ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:34.634922 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3ad80-50fd-4da7-ab6c-d8a95fbe73ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:34.636846 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-09d3ad80-50fd-4da7-ab6c-d8a95fbe73ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:34.645400 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:34.657005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1105/2228] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:34.666196 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e0d05-9690-40a1-830f-02cb5d579cf7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:34.667699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8d5e0d05-9690-40a1-830f-02cb5d579cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:34.738025 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d3ad80-50fd-4da7-ab6c-d8a95fbe73ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:34.739500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d3ad80-50fd-4da7-ab6c-d8a95fbe73ff service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:34.740794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e0d05-9690-40a1-830f-02cb5d579cf7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:34.745053 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7625c-8056-465e-a428-ef511288faf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:34.745812 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7625c-8056-465e-a428-ef511288faf6 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:34.746166 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7625c-8056-465e-a428-ef511288faf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:34.746665 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7625c-8056-465e-a428-ef511288faf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:34.750224 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1124/2229] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:34.759118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7625c-8056-465e-a428-ef511288faf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:34.760584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7625c-8056-465e-a428-ef511288faf6 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:34.764881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:34.777762 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1106/2230] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:35.223866 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3ad80-50fd-4da7-ab6c-d8a95fbe73ff service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:35.225455 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-09d3ad80-50fd-4da7-ab6c-d8a95fbe73ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:35.253799 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3ad80-50fd-4da7-ab6c-d8a95fbe73ff service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:35.260746 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25750d26-997c-4586-ac8c-c95f1b298707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:35.262677 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25750d26-997c-4586-ac8c-c95f1b298707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:35.263121 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25750d26-997c-4586-ac8c-c95f1b298707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:35.263952 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25750d26-997c-4586-ac8c-c95f1b298707 admin admin] PATH_INFO: `/v3/projects/0ca03c0d189a402389e51c76437bae45` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:35.266689 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25750d26-997c-4586-ac8c-c95f1b298707 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ca03c0d189a402389e51c76437bae45)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:35.269694 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25750d26-997c-4586-ac8c-c95f1b298707 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:35.343363 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-25750d26-997c-4586-ac8c-c95f1b298707 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ca03c0d189a402389e51c76437bae45', 'request_id': 'req-25750d26-997c-4586-ac8c-c95f1b298707'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:35.353703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-25750d26-997c-4586-ac8c-c95f1b298707 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ca03c0d189a402389e51c76437bae45', 'request_id': 'req-25750d26-997c-4586-ac8c-c95f1b298707'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:35.364361 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-25750d26-997c-4586-ac8c-c95f1b298707 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ca03c0d189a402389e51c76437bae45', 'request_id': 'req-25750d26-997c-4586-ac8c-c95f1b298707'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:35.381123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-25750d26-997c-4586-ac8c-c95f1b298707 admin admin] The token cache is being invalidate because project 0ca03c0d189a402389e51c76437bae45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:30:35.381520 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-25750d26-997c-4586-ac8c-c95f1b298707 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:35.382554 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-25750d26-997c-4586-ac8c-c95f1b298707 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:35.393138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1125/2231] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:30:35 2022] DELETE /identity/v3/projects/0ca03c0d189a402389e51c76437bae45 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:35.397272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7625c-8056-465e-a428-ef511288faf6 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:35.409739 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7625c-8056-465e-a428-ef511288faf6 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:35.481482 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279528cd-ad54-4dd2-a293-fc670489a95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:35.484470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279528cd-ad54-4dd2-a293-fc670489a95e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:35.484470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279528cd-ad54-4dd2-a293-fc670489a95e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:35.484470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279528cd-ad54-4dd2-a293-fc670489a95e admin admin] PATH_INFO: `/v3/users/ef97ce69f08d4c6b926128de7e936bc9` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:35.485821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279528cd-ad54-4dd2-a293-fc670489a95e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef97ce69f08d4c6b926128de7e936bc9)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:35.487802 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279528cd-ad54-4dd2-a293-fc670489a95e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:35.549126 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-279528cd-ad54-4dd2-a293-fc670489a95e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef97ce69f08d4c6b926128de7e936bc9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:35.549859 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-279528cd-ad54-4dd2-a293-fc670489a95e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef97ce69f08d4c6b926128de7e936bc9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:35.562325 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-279528cd-ad54-4dd2-a293-fc670489a95e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef97ce69f08d4c6b926128de7e936bc9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:35.572126 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-279528cd-ad54-4dd2-a293-fc670489a95e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef97ce69f08d4c6b926128de7e936bc9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:35.572917 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-279528cd-ad54-4dd2-a293-fc670489a95e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef97ce69f08d4c6b926128de7e936bc9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:35.588670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1107/2232] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:30:35 2022] DELETE /identity/v3/users/ef97ce69f08d4c6b926128de7e936bc9 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:35.878068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25750d26-997c-4586-ac8c-c95f1b298707 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:35.879330 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-25750d26-997c-4586-ac8c-c95f1b298707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:35.897544 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25750d26-997c-4586-ac8c-c95f1b298707 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:35.969299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df319c-c2e1-4b51-acec-c2b0cfbb3383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:35.970204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df319c-c2e1-4b51-acec-c2b0cfbb3383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:35.970489 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df319c-c2e1-4b51-acec-c2b0cfbb3383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:35.970820 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df319c-c2e1-4b51-acec-c2b0cfbb3383 admin admin] PATH_INFO: `/v3/projects/06668268a675484aa22e45a49de74291` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:35.971952 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6df319c-c2e1-4b51-acec-c2b0cfbb3383 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06668268a675484aa22e45a49de74291)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:35.974794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6df319c-c2e1-4b51-acec-c2b0cfbb3383 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:36.026521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:36.038097 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e6df319c-c2e1-4b51-acec-c2b0cfbb3383 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06668268a675484aa22e45a49de74291', 'request_id': 'req-e6df319c-c2e1-4b51-acec-c2b0cfbb3383'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:36.046806 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e6df319c-c2e1-4b51-acec-c2b0cfbb3383 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06668268a675484aa22e45a49de74291', 'request_id': 'req-e6df319c-c2e1-4b51-acec-c2b0cfbb3383'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:36.052968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e6df319c-c2e1-4b51-acec-c2b0cfbb3383 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06668268a675484aa22e45a49de74291', 'request_id': 'req-e6df319c-c2e1-4b51-acec-c2b0cfbb3383'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:36.066709 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e6df319c-c2e1-4b51-acec-c2b0cfbb3383 admin admin] The token cache is being invalidate because project 06668268a675484aa22e45a49de74291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:30:36.069732 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e6df319c-c2e1-4b51-acec-c2b0cfbb3383 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:36.070233 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e6df319c-c2e1-4b51-acec-c2b0cfbb3383 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:36.072271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1126/2233] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:30:35 2022] DELETE /identity/v3/projects/06668268a675484aa22e45a49de74291 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:36.076146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279528cd-ad54-4dd2-a293-fc670489a95e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:36.078534 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-279528cd-ad54-4dd2-a293-fc670489a95e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:36.091627 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279528cd-ad54-4dd2-a293-fc670489a95e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:36.160174 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a8ea1-8dfe-465f-ba67-7283863aaf5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:36.161300 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a8ea1-8dfe-465f-ba67-7283863aaf5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:36.161617 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a8ea1-8dfe-465f-ba67-7283863aaf5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:36.162533 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a8ea1-8dfe-465f-ba67-7283863aaf5a admin admin] PATH_INFO: `/v3/users/c811254ec65843b9af78f8c64f621536` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:36.163940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a8ea1-8dfe-465f-ba67-7283863aaf5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c811254ec65843b9af78f8c64f621536)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:36.166633 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a8ea1-8dfe-465f-ba67-7283863aaf5a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:36.205780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:36.225225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b2671-8221-4eb0-81d7-cd444282d878 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:36.225516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b2671-8221-4eb0-81d7-cd444282d878 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:36.225834 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b2671-8221-4eb0-81d7-cd444282d878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:36.240805 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fd7a8ea1-8dfe-465f-ba67-7283863aaf5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c811254ec65843b9af78f8c64f621536'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:36.241623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fd7a8ea1-8dfe-465f-ba67-7283863aaf5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c811254ec65843b9af78f8c64f621536'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:36.245948 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8b8b2671-8221-4eb0-81d7-cd444282d878 None None] MFA Rules not processed for user `374555cda1b4424894093ebcc51cd651`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:36.247166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8b8b2671-8221-4eb0-81d7-cd444282d878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:36.257091 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fd7a8ea1-8dfe-465f-ba67-7283863aaf5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c811254ec65843b9af78f8c64f621536'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:36.270028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fd7a8ea1-8dfe-465f-ba67-7283863aaf5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c811254ec65843b9af78f8c64f621536'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:36.271819 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-fd7a8ea1-8dfe-465f-ba67-7283863aaf5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c811254ec65843b9af78f8c64f621536'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:36.292355 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1108/2234] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:30:36 2022] DELETE /identity/v3/users/c811254ec65843b9af78f8c64f621536 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:36.347138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1127/2235] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:30:36.642671 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a8ea1-8dfe-465f-ba67-7283863aaf5a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:36.644563 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fd7a8ea1-8dfe-465f-ba67-7283863aaf5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:36.757361 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a8ea1-8dfe-465f-ba67-7283863aaf5a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:36.762400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d04527-e079-41e1-aa1d-3d74551bfbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:36.763585 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d04527-e079-41e1-aa1d-3d74551bfbd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:36.764608 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d04527-e079-41e1-aa1d-3d74551bfbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:36.765118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d04527-e079-41e1-aa1d-3d74551bfbd5 admin admin] PATH_INFO: `/v3/projects/d582c012fc614a9f87a0d470d6467375` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:36.766726 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d04527-e079-41e1-aa1d-3d74551bfbd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d582c012fc614a9f87a0d470d6467375)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:36.768751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d04527-e079-41e1-aa1d-3d74551bfbd5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:36.824517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e7d04527-e079-41e1-aa1d-3d74551bfbd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd582c012fc614a9f87a0d470d6467375', 'request_id': 'req-e7d04527-e079-41e1-aa1d-3d74551bfbd5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:36.831170 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e7d04527-e079-41e1-aa1d-3d74551bfbd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd582c012fc614a9f87a0d470d6467375', 'request_id': 'req-e7d04527-e079-41e1-aa1d-3d74551bfbd5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:36.837095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e7d04527-e079-41e1-aa1d-3d74551bfbd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd582c012fc614a9f87a0d470d6467375', 'request_id': 'req-e7d04527-e079-41e1-aa1d-3d74551bfbd5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:36.853571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e7d04527-e079-41e1-aa1d-3d74551bfbd5 admin admin] The token cache is being invalidate because project d582c012fc614a9f87a0d470d6467375 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:30:36.853957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e7d04527-e079-41e1-aa1d-3d74551bfbd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:36.854603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e7d04527-e079-41e1-aa1d-3d74551bfbd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:36.856287 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1109/2236] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:30:36 2022] DELETE /identity/v3/projects/d582c012fc614a9f87a0d470d6467375 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:36.862586 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd82a1-cdb1-4842-a771-76977c3540ad None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:36.862882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd82a1-cdb1-4842-a771-76977c3540ad None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:36.863172 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd82a1-cdb1-4842-a771-76977c3540ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:36.882367 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-94fd82a1-cdb1-4842-a771-76977c3540ad None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:36.883578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-94fd82a1-cdb1-4842-a771-76977c3540ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:36.946960 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1128/2237] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:36.955401 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c3429-d562-4795-b85f-0d40e9f6b0fd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:36.955913 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c3429-d562-4795-b85f-0d40e9f6b0fd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:36.956204 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c3429-d562-4795-b85f-0d40e9f6b0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:36.977404 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4e2c3429-d562-4795-b85f-0d40e9f6b0fd None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:36.978597 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4e2c3429-d562-4795-b85f-0d40e9f6b0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:36.988134 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1110/2238] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:36.992124 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd82a1-cdb1-4842-a771-76977c3540ad None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:37.009858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd82a1-cdb1-4842-a771-76977c3540ad None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:37.015041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc0698-4069-464c-a2bb-f5c4ecbc96cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:37.015822 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc0698-4069-464c-a2bb-f5c4ecbc96cf admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:37.016302 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc0698-4069-464c-a2bb-f5c4ecbc96cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:37.016510 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc0698-4069-464c-a2bb-f5c4ecbc96cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:37.017256 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc0698-4069-464c-a2bb-f5c4ecbc96cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:30:37.017562 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc0698-4069-464c-a2bb-f5c4ecbc96cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:37.019252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc0698-4069-464c-a2bb-f5c4ecbc96cf admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:37.033408 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1129/2239] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:30:37.039272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c3429-d562-4795-b85f-0d40e9f6b0fd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:37.049691 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c3429-d562-4795-b85f-0d40e9f6b0fd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:37.054586 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa229e8-edb5-423d-bcbf-df6d6cfc978c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:37.055220 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa229e8-edb5-423d-bcbf-df6d6cfc978c admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:37.055501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa229e8-edb5-423d-bcbf-df6d6cfc978c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:37.055772 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa229e8-edb5-423d-bcbf-df6d6cfc978c admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:37.056473 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa229e8-edb5-423d-bcbf-df6d6cfc978c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:37.058798 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa229e8-edb5-423d-bcbf-df6d6cfc978c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:37.080385 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:37.093267 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1111/2240] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:30:37 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:30:37.094278 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc0698-4069-464c-a2bb-f5c4ecbc96cf admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:37.104540 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc0698-4069-464c-a2bb-f5c4ecbc96cf admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:37.167402 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:37.179081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa606d3-4a01-44a8-8e17-c5cc2da85601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:37.179785 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa606d3-4a01-44a8-8e17-c5cc2da85601 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:37.180077 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa606d3-4a01-44a8-8e17-c5cc2da85601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:37.180371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa606d3-4a01-44a8-8e17-c5cc2da85601 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:37.181073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa606d3-4a01-44a8-8e17-c5cc2da85601 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:37.183102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa606d3-4a01-44a8-8e17-c5cc2da85601 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:37.204665 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1130/2241] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:30:37 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:30:37.208884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa229e8-edb5-423d-bcbf-df6d6cfc978c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:37.219549 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa229e8-edb5-423d-bcbf-df6d6cfc978c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:37.226531 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c43b8b-4f3a-4fd0-9bd5-01f24376679d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:37.227200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c43b8b-4f3a-4fd0-9bd5-01f24376679d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:37.227535 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c43b8b-4f3a-4fd0-9bd5-01f24376679d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:37.227763 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c43b8b-4f3a-4fd0-9bd5-01f24376679d admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:37.228499 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c43b8b-4f3a-4fd0-9bd5-01f24376679d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:37.230149 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c43b8b-4f3a-4fd0-9bd5-01f24376679d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:37.245636 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1112/2242] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:37.247664 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa606d3-4a01-44a8-8e17-c5cc2da85601 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:37.259698 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa606d3-4a01-44a8-8e17-c5cc2da85601 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:37.265172 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04935c98-946d-4230-b6fa-8371f9b5dd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:37.266424 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04935c98-946d-4230-b6fa-8371f9b5dd80 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:37.266718 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04935c98-946d-4230-b6fa-8371f9b5dd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:37.267130 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04935c98-946d-4230-b6fa-8371f9b5dd80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:37.267873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04935c98-946d-4230-b6fa-8371f9b5dd80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:37.269474 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04935c98-946d-4230-b6fa-8371f9b5dd80 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:37.280036 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1131/2243] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:37.284174 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c43b8b-4f3a-4fd0-9bd5-01f24376679d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:37.295341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c43b8b-4f3a-4fd0-9bd5-01f24376679d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:37.300124 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68509a1b-44fd-415b-8325-2a77a0824e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:37.301082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68509a1b-44fd-415b-8325-2a77a0824e61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:37.301396 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68509a1b-44fd-415b-8325-2a77a0824e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:37.301567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68509a1b-44fd-415b-8325-2a77a0824e61 admin admin] PATH_INFO: `/v3/projects/35765acef1844ca1a2941b765987082b/users/7eff70262de1440f874e2ca44230c5ab/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:37.324435 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68509a1b-44fd-415b-8325-2a77a0824e61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35765acef1844ca1a2941b765987082b, user_id=7eff70262de1440f874e2ca44230c5ab, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:37.334203 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68509a1b-44fd-415b-8325-2a77a0824e61 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:37.351611 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1113/2244] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:30:37 2022] PUT /identity/v3/projects/35765acef1844ca1a2941b765987082b/users/7eff70262de1440f874e2ca44230c5ab/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:37.363709 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a8c9f-cc6c-49e6-8877-44709dd6ff6a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:37.364185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a8c9f-cc6c-49e6-8877-44709dd6ff6a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:37.364289 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a8c9f-cc6c-49e6-8877-44709dd6ff6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:37.409263 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f70a8c9f-cc6c-49e6-8877-44709dd6ff6a None None] MFA Rules not processed for user `7eff70262de1440f874e2ca44230c5ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:37.411089 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f70a8c9f-cc6c-49e6-8877-44709dd6ff6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:37.490970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1132/2245] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:37.506916 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68509a1b-44fd-415b-8325-2a77a0824e61 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:37.509037 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-68509a1b-44fd-415b-8325-2a77a0824e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:37.609250 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68509a1b-44fd-415b-8325-2a77a0824e61 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:37.619310 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d124c-fef1-470a-9a16-b831601e15f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:37.620943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d124c-fef1-470a-9a16-b831601e15f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:37.621419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d124c-fef1-470a-9a16-b831601e15f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:37.625996 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d124c-fef1-470a-9a16-b831601e15f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:37.655515 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d124c-fef1-470a-9a16-b831601e15f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:37.657893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d124c-fef1-470a-9a16-b831601e15f6 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:37.671188 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1114/2246] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:30:41.311372 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a8c9f-cc6c-49e6-8877-44709dd6ff6a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:41.383547 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a8c9f-cc6c-49e6-8877-44709dd6ff6a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:41.388061 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb0300-826d-4fa8-821d-c9677986a9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:41.388925 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb0300-826d-4fa8-821d-c9677986a9e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:41.389750 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb0300-826d-4fa8-821d-c9677986a9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:41.390491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb0300-826d-4fa8-821d-c9677986a9e8 admin admin] PATH_INFO: `/v3/users/7eff70262de1440f874e2ca44230c5ab` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:41.391693 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fb0300-826d-4fa8-821d-c9677986a9e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7eff70262de1440f874e2ca44230c5ab)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:41.393664 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fb0300-826d-4fa8-821d-c9677986a9e8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:41.439718 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d5fb0300-826d-4fa8-821d-c9677986a9e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7eff70262de1440f874e2ca44230c5ab'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:41.451094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:41.453504 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d5fb0300-826d-4fa8-821d-c9677986a9e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7eff70262de1440f874e2ca44230c5ab'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:41.454741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d5fb0300-826d-4fa8-821d-c9677986a9e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7eff70262de1440f874e2ca44230c5ab'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:41.463456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d5fb0300-826d-4fa8-821d-c9677986a9e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7eff70262de1440f874e2ca44230c5ab'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:41.464268 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d5fb0300-826d-4fa8-821d-c9677986a9e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7eff70262de1440f874e2ca44230c5ab'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:41.479639 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1133/2247] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:30:41 2022] DELETE /identity/v3/users/7eff70262de1440f874e2ca44230c5ab => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:41.484520 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50f450-1162-40b4-a2d0-a5e4032ed468 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:41.485005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50f450-1162-40b4-a2d0-a5e4032ed468 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:41.485152 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50f450-1162-40b4-a2d0-a5e4032ed468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:41.509543 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3a50f450-1162-40b4-a2d0-a5e4032ed468 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:41.510847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3a50f450-1162-40b4-a2d0-a5e4032ed468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:41.585967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1115/2248] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:41.596730 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb0300-826d-4fa8-821d-c9677986a9e8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:41.598751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d5fb0300-826d-4fa8-821d-c9677986a9e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:41.668081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb0300-826d-4fa8-821d-c9677986a9e8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:41.672175 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc99c7c-9c73-47f2-bd77-1c9bc5d9ad6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:41.673018 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc99c7c-9c73-47f2-bd77-1c9bc5d9ad6e service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:41.673311 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc99c7c-9c73-47f2-bd77-1c9bc5d9ad6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:41.673636 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc99c7c-9c73-47f2-bd77-1c9bc5d9ad6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:41.683712 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc99c7c-9c73-47f2-bd77-1c9bc5d9ad6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:41.685416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc99c7c-9c73-47f2-bd77-1c9bc5d9ad6e service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:41.699748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1134/2249] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:42.058098 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50f450-1162-40b4-a2d0-a5e4032ed468 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:42.059755 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3a50f450-1162-40b4-a2d0-a5e4032ed468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:42.075990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50f450-1162-40b4-a2d0-a5e4032ed468 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:42.081876 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f835988-e1e5-4c79-b177-e62cd4674108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:42.082938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f835988-e1e5-4c79-b177-e62cd4674108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:42.083320 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f835988-e1e5-4c79-b177-e62cd4674108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:42.083742 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f835988-e1e5-4c79-b177-e62cd4674108 admin admin] PATH_INFO: `/v3/projects/35765acef1844ca1a2941b765987082b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:42.085028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f835988-e1e5-4c79-b177-e62cd4674108 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35765acef1844ca1a2941b765987082b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:42.087312 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f835988-e1e5-4c79-b177-e62cd4674108 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:42.135886 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9f835988-e1e5-4c79-b177-e62cd4674108 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35765acef1844ca1a2941b765987082b', 'request_id': 'req-9f835988-e1e5-4c79-b177-e62cd4674108'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:42.142877 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9f835988-e1e5-4c79-b177-e62cd4674108 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35765acef1844ca1a2941b765987082b', 'request_id': 'req-9f835988-e1e5-4c79-b177-e62cd4674108'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:42.147500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:42.148827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9f835988-e1e5-4c79-b177-e62cd4674108 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35765acef1844ca1a2941b765987082b', 'request_id': 'req-9f835988-e1e5-4c79-b177-e62cd4674108'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:42.163709 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9f835988-e1e5-4c79-b177-e62cd4674108 admin admin] The token cache is being invalidate because project 35765acef1844ca1a2941b765987082b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:30:42.163998 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9f835988-e1e5-4c79-b177-e62cd4674108 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:42.164753 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9f835988-e1e5-4c79-b177-e62cd4674108 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:42.166406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1116/2250] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:30:42 2022] DELETE /identity/v3/projects/35765acef1844ca1a2941b765987082b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:42.171804 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f58747-3094-4033-b748-d4aeb61884d1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:42.172530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f58747-3094-4033-b748-d4aeb61884d1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:42.173007 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f58747-3094-4033-b748-d4aeb61884d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:42.191620 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e4f58747-3094-4033-b748-d4aeb61884d1 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:42.192854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e4f58747-3094-4033-b748-d4aeb61884d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:42.271126 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1135/2251] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:42.279376 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217a1e5-4880-47ee-8b55-7be2a8d7089d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:42.280290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217a1e5-4880-47ee-8b55-7be2a8d7089d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:42.281421 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217a1e5-4880-47ee-8b55-7be2a8d7089d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:42.301645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e217a1e5-4880-47ee-8b55-7be2a8d7089d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:42.303182 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e217a1e5-4880-47ee-8b55-7be2a8d7089d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:42.320387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1117/2252] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:30:42.324770 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f58747-3094-4033-b748-d4aeb61884d1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:42.334781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f58747-3094-4033-b748-d4aeb61884d1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:42.339430 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6683853-171b-4fd2-89c2-4ecb8e220403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:42.340084 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6683853-171b-4fd2-89c2-4ecb8e220403 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:42.340533 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6683853-171b-4fd2-89c2-4ecb8e220403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:42.340633 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6683853-171b-4fd2-89c2-4ecb8e220403 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:42.341346 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6683853-171b-4fd2-89c2-4ecb8e220403 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:30:42.341687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6683853-171b-4fd2-89c2-4ecb8e220403 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:42.344087 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6683853-171b-4fd2-89c2-4ecb8e220403 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:42.368518 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217a1e5-4880-47ee-8b55-7be2a8d7089d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:42.371327 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217a1e5-4880-47ee-8b55-7be2a8d7089d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:42.372255 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1136/2253] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:30:42.377361 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63bd101-613a-42e6-903c-b42b86af3536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:42.378099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bd101-613a-42e6-903c-b42b86af3536 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:42.378345 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bd101-613a-42e6-903c-b42b86af3536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:42.378621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bd101-613a-42e6-903c-b42b86af3536 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:42.379309 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63bd101-613a-42e6-903c-b42b86af3536 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:42.381252 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63bd101-613a-42e6-903c-b42b86af3536 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:42.412588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1118/2254] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:30:42 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:30:42.413834 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6683853-171b-4fd2-89c2-4ecb8e220403 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:42.424720 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6683853-171b-4fd2-89c2-4ecb8e220403 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:42.484359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d55bd-4636-49c7-bc4a-7fd147cefdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:42.484920 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d55bd-4636-49c7-bc4a-7fd147cefdae admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:42.485210 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d55bd-4636-49c7-bc4a-7fd147cefdae admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:42.485489 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d55bd-4636-49c7-bc4a-7fd147cefdae admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:42.486203 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d55bd-4636-49c7-bc4a-7fd147cefdae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:42.488119 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d55bd-4636-49c7-bc4a-7fd147cefdae admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:42.504652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:42.515904 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1137/2255] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:30:42 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:30:42.516355 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63bd101-613a-42e6-903c-b42b86af3536 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:42.528356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63bd101-613a-42e6-903c-b42b86af3536 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:42.533902 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b4b25-5cfc-4780-8799-36fa7b4b6f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:42.534591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b4b25-5cfc-4780-8799-36fa7b4b6f71 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:42.534870 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b4b25-5cfc-4780-8799-36fa7b4b6f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:42.535141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b4b25-5cfc-4780-8799-36fa7b4b6f71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:42.535868 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b4b25-5cfc-4780-8799-36fa7b4b6f71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:42.537458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b4b25-5cfc-4780-8799-36fa7b4b6f71 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:42.548881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1119/2256] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:42.552908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d55bd-4636-49c7-bc4a-7fd147cefdae admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:42.566749 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d55bd-4636-49c7-bc4a-7fd147cefdae admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:42.574620 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd933d9a-380d-42f6-8ad9-419cc30ae78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:42.575297 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd933d9a-380d-42f6-8ad9-419cc30ae78f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:42.575588 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd933d9a-380d-42f6-8ad9-419cc30ae78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:42.575875 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd933d9a-380d-42f6-8ad9-419cc30ae78f admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:42.577243 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd933d9a-380d-42f6-8ad9-419cc30ae78f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:42.579083 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd933d9a-380d-42f6-8ad9-419cc30ae78f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:42.595978 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1138/2257] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:42.596730 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b4b25-5cfc-4780-8799-36fa7b4b6f71 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:42.607717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b4b25-5cfc-4780-8799-36fa7b4b6f71 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:42.613875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb492f-a08d-4d57-9cad-dea359932211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:42.615077 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb492f-a08d-4d57-9cad-dea359932211 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:42.615377 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb492f-a08d-4d57-9cad-dea359932211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:42.615663 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb492f-a08d-4d57-9cad-dea359932211 admin admin] PATH_INFO: `/v3/projects/e6464742299a4dd7a79209c0a9932ddb/users/87d3d8e9b81e4228b3a7f6934df93286/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:42.638798 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb492f-a08d-4d57-9cad-dea359932211 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6464742299a4dd7a79209c0a9932ddb, user_id=87d3d8e9b81e4228b3a7f6934df93286, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:42.644244 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb492f-a08d-4d57-9cad-dea359932211 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:42.661832 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1120/2258] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:30:42 2022] PUT /identity/v3/projects/e6464742299a4dd7a79209c0a9932ddb/users/87d3d8e9b81e4228b3a7f6934df93286/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:30:42.672293 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10106e-f9a5-415b-9210-48fbe5a10a75 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:42.672588 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10106e-f9a5-415b-9210-48fbe5a10a75 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:42.672879 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10106e-f9a5-415b-9210-48fbe5a10a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:42.727277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ac10106e-f9a5-415b-9210-48fbe5a10a75 None None] MFA Rules not processed for user `87d3d8e9b81e4228b3a7f6934df93286`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:42.728984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ac10106e-f9a5-415b-9210-48fbe5a10a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:42.807890 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1139/2259] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:42.820451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb492f-a08d-4d57-9cad-dea359932211 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:42.823070 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-09bb492f-a08d-4d57-9cad-dea359932211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:42.907844 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb492f-a08d-4d57-9cad-dea359932211 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:42.912526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c5e45-6329-4420-a536-f7205b910308 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:42.913186 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c5e45-6329-4420-a536-f7205b910308 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:42.913467 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c5e45-6329-4420-a536-f7205b910308 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:42.913775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c5e45-6329-4420-a536-f7205b910308 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:42.928475 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c5e45-6329-4420-a536-f7205b910308 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:42.930789 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c5e45-6329-4420-a536-f7205b910308 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:42.952956 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1121/2260] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:52.804452 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=37 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:52.948799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=56,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:57.450313 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10106e-f9a5-415b-9210-48fbe5a10a75 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:57.557572 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10106e-f9a5-415b-9210-48fbe5a10a75 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:57.563389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de09bc-aa07-4ad3-819b-01ab23881c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:57.564284 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de09bc-aa07-4ad3-819b-01ab23881c74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:57.564567 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de09bc-aa07-4ad3-819b-01ab23881c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:57.564844 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de09bc-aa07-4ad3-819b-01ab23881c74 admin admin] PATH_INFO: `/v3/users/87d3d8e9b81e4228b3a7f6934df93286` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:57.566077 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1de09bc-aa07-4ad3-819b-01ab23881c74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87d3d8e9b81e4228b3a7f6934df93286)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:57.568282 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1de09bc-aa07-4ad3-819b-01ab23881c74 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:57.619263 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f1de09bc-aa07-4ad3-819b-01ab23881c74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87d3d8e9b81e4228b3a7f6934df93286'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:57.631254 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f1de09bc-aa07-4ad3-819b-01ab23881c74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87d3d8e9b81e4228b3a7f6934df93286'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:57.631893 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f1de09bc-aa07-4ad3-819b-01ab23881c74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87d3d8e9b81e4228b3a7f6934df93286'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:57.638544 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f1de09bc-aa07-4ad3-819b-01ab23881c74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87d3d8e9b81e4228b3a7f6934df93286'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:57.639166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f1de09bc-aa07-4ad3-819b-01ab23881c74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87d3d8e9b81e4228b3a7f6934df93286'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:57.651899 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1140/2261] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:30:57 2022] DELETE /identity/v3/users/87d3d8e9b81e4228b3a7f6934df93286 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:57.657459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64538f5b-5749-4c9a-a38e-1ce1993a9bc6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:57.658014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64538f5b-5749-4c9a-a38e-1ce1993a9bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:57.658117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64538f5b-5749-4c9a-a38e-1ce1993a9bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:57.713104 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-64538f5b-5749-4c9a-a38e-1ce1993a9bc6 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:57.714436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-64538f5b-5749-4c9a-a38e-1ce1993a9bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:57.773892 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1122/2262] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:57.784129 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de09bc-aa07-4ad3-819b-01ab23881c74 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:57.785543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f1de09bc-aa07-4ad3-819b-01ab23881c74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:57.830706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:57.899559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de09bc-aa07-4ad3-819b-01ab23881c74 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:57.903851 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77f168-ec25-4bd8-8dec-5fc5c6d14900 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:57.904517 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77f168-ec25-4bd8-8dec-5fc5c6d14900 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:57.904802 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77f168-ec25-4bd8-8dec-5fc5c6d14900 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:57.905628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77f168-ec25-4bd8-8dec-5fc5c6d14900 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:57.916074 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a77f168-ec25-4bd8-8dec-5fc5c6d14900 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:57.917384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a77f168-ec25-4bd8-8dec-5fc5c6d14900 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:57.928742 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1141/2263] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:58.506423 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64538f5b-5749-4c9a-a38e-1ce1993a9bc6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:58.508002 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-64538f5b-5749-4c9a-a38e-1ce1993a9bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:58.521969 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64538f5b-5749-4c9a-a38e-1ce1993a9bc6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:58.526594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62864ab8-4815-476e-a6bd-ae64aef9b853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:58.527658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62864ab8-4815-476e-a6bd-ae64aef9b853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:58.527955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62864ab8-4815-476e-a6bd-ae64aef9b853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:58.528236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62864ab8-4815-476e-a6bd-ae64aef9b853 admin admin] PATH_INFO: `/v3/projects/e6464742299a4dd7a79209c0a9932ddb` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:58.529557 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62864ab8-4815-476e-a6bd-ae64aef9b853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6464742299a4dd7a79209c0a9932ddb)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:58.532433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62864ab8-4815-476e-a6bd-ae64aef9b853 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:58.582686 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-62864ab8-4815-476e-a6bd-ae64aef9b853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6464742299a4dd7a79209c0a9932ddb', 'request_id': 'req-62864ab8-4815-476e-a6bd-ae64aef9b853'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:58.588823 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-62864ab8-4815-476e-a6bd-ae64aef9b853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6464742299a4dd7a79209c0a9932ddb', 'request_id': 'req-62864ab8-4815-476e-a6bd-ae64aef9b853'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:58.595651 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-62864ab8-4815-476e-a6bd-ae64aef9b853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6464742299a4dd7a79209c0a9932ddb', 'request_id': 'req-62864ab8-4815-476e-a6bd-ae64aef9b853'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:58.607202 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-62864ab8-4815-476e-a6bd-ae64aef9b853 admin admin] The token cache is being invalidate because project e6464742299a4dd7a79209c0a9932ddb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:30:58.607484 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-62864ab8-4815-476e-a6bd-ae64aef9b853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:58.608106 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-62864ab8-4815-476e-a6bd-ae64aef9b853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:58.609955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1123/2264] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:30:58 2022] DELETE /identity/v3/projects/e6464742299a4dd7a79209c0a9932ddb => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:31:07.461026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77f168-ec25-4bd8-8dec-5fc5c6d14900 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:07.462484 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6a77f168-ec25-4bd8-8dec-5fc5c6d14900 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:07.476343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77f168-ec25-4bd8-8dec-5fc5c6d14900 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:07.532390 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01b0ff-0cd2-4f6f-abce-122fc5264caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:07.533244 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01b0ff-0cd2-4f6f-abce-122fc5264caa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:07.533526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01b0ff-0cd2-4f6f-abce-122fc5264caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:07.533835 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01b0ff-0cd2-4f6f-abce-122fc5264caa admin admin] PATH_INFO: `/v3/users/c6edb603589b4b1984b3b2a9df6c8151` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:07.535014 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a01b0ff-0cd2-4f6f-abce-122fc5264caa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6edb603589b4b1984b3b2a9df6c8151)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:07.536942 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a01b0ff-0cd2-4f6f-abce-122fc5264caa admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:07.582059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8a01b0ff-0cd2-4f6f-abce-122fc5264caa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6edb603589b4b1984b3b2a9df6c8151'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:07.595522 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8a01b0ff-0cd2-4f6f-abce-122fc5264caa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6edb603589b4b1984b3b2a9df6c8151'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:07.596230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8a01b0ff-0cd2-4f6f-abce-122fc5264caa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6edb603589b4b1984b3b2a9df6c8151'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:07.603041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8a01b0ff-0cd2-4f6f-abce-122fc5264caa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6edb603589b4b1984b3b2a9df6c8151'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:07.603680 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-8a01b0ff-0cd2-4f6f-abce-122fc5264caa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6edb603589b4b1984b3b2a9df6c8151'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:07.616762 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1142/2265] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:31:07 2022] DELETE /identity/v3/users/c6edb603589b4b1984b3b2a9df6c8151 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:31:07.621263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca11a25-2144-4c12-a889-ee21b830f37d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:07.621767 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca11a25-2144-4c12-a889-ee21b830f37d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:07.621863 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca11a25-2144-4c12-a889-ee21b830f37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:07.640790 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8ca11a25-2144-4c12-a889-ee21b830f37d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:07.642053 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8ca11a25-2144-4c12-a889-ee21b830f37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:07.697797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:07.708747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1124/2266] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:31:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:07.718730 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01b0ff-0cd2-4f6f-abce-122fc5264caa admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:07.720193 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8a01b0ff-0cd2-4f6f-abce-122fc5264caa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:07.748608 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:07.779762 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01b0ff-0cd2-4f6f-abce-122fc5264caa admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:07.784312 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66291d-4305-4446-add3-53bd6502c2d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:07.784950 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66291d-4305-4446-add3-53bd6502c2d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:07.785238 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66291d-4305-4446-add3-53bd6502c2d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:07.785519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66291d-4305-4446-add3-53bd6502c2d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:07.794729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66291d-4305-4446-add3-53bd6502c2d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:07.796764 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66291d-4305-4446-add3-53bd6502c2d4 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:07.806304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1143/2267] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:31:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:31:08.079845 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca11a25-2144-4c12-a889-ee21b830f37d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:08.081279 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8ca11a25-2144-4c12-a889-ee21b830f37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:08.096212 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca11a25-2144-4c12-a889-ee21b830f37d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:08.101822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c99b9-58ba-4d40-a54a-c2a13204ba4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:08.102699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c99b9-58ba-4d40-a54a-c2a13204ba4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:08.102985 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c99b9-58ba-4d40-a54a-c2a13204ba4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:08.103262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c99b9-58ba-4d40-a54a-c2a13204ba4a admin admin] PATH_INFO: `/v3/projects/0b32a883f9ee4186b559190a6cddf33f` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:08.104519 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c99b9-58ba-4d40-a54a-c2a13204ba4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b32a883f9ee4186b559190a6cddf33f)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:08.106500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c99b9-58ba-4d40-a54a-c2a13204ba4a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:08.156493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1a4c99b9-58ba-4d40-a54a-c2a13204ba4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b32a883f9ee4186b559190a6cddf33f', 'request_id': 'req-1a4c99b9-58ba-4d40-a54a-c2a13204ba4a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:08.162784 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1a4c99b9-58ba-4d40-a54a-c2a13204ba4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b32a883f9ee4186b559190a6cddf33f', 'request_id': 'req-1a4c99b9-58ba-4d40-a54a-c2a13204ba4a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:08.168548 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1a4c99b9-58ba-4d40-a54a-c2a13204ba4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b32a883f9ee4186b559190a6cddf33f', 'request_id': 'req-1a4c99b9-58ba-4d40-a54a-c2a13204ba4a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:08.181822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1a4c99b9-58ba-4d40-a54a-c2a13204ba4a admin admin] The token cache is being invalidate because project 0b32a883f9ee4186b559190a6cddf33f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:31:08.182261 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1a4c99b9-58ba-4d40-a54a-c2a13204ba4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:08.182892 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1a4c99b9-58ba-4d40-a54a-c2a13204ba4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:08.184410 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1125/2268] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:31:08 2022] DELETE /identity/v3/projects/0b32a883f9ee4186b559190a6cddf33f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:31:08.189981 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94b515-f311-4c14-a3f8-070fd7bc6b16 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:08.190337 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94b515-f311-4c14-a3f8-070fd7bc6b16 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:08.190699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94b515-f311-4c14-a3f8-070fd7bc6b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:08.239531 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1c94b515-f311-4c14-a3f8-070fd7bc6b16 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:08.240783 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1c94b515-f311-4c14-a3f8-070fd7bc6b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:08.314814 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1144/2269] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:31:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:08.322450 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b141208-e71b-4b0e-ba2f-4f6e4d8a2850 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:08.322737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b141208-e71b-4b0e-ba2f-4f6e4d8a2850 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:08.323016 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b141208-e71b-4b0e-ba2f-4f6e4d8a2850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:08.347342 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3b141208-e71b-4b0e-ba2f-4f6e4d8a2850 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:08.349355 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3b141208-e71b-4b0e-ba2f-4f6e4d8a2850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:08.362685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1126/2270] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:31:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:08.364017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94b515-f311-4c14-a3f8-070fd7bc6b16 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:08.380634 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94b515-f311-4c14-a3f8-070fd7bc6b16 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:08.387887 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1d253-4472-4f6f-87af-99c1b014c4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:08.388515 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1d253-4472-4f6f-87af-99c1b014c4e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:08.388799 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1d253-4472-4f6f-87af-99c1b014c4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:08.389078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1d253-4472-4f6f-87af-99c1b014c4e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:08.390444 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c1d253-4472-4f6f-87af-99c1b014c4e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:31:08.390761 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c1d253-4472-4f6f-87af-99c1b014c4e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:08.392385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c1d253-4472-4f6f-87af-99c1b014c4e6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:08.404887 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1145/2271] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:31:08.409299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b141208-e71b-4b0e-ba2f-4f6e4d8a2850 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:08.425393 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b141208-e71b-4b0e-ba2f-4f6e4d8a2850 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:08.430333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e6585-5420-4d9e-8888-1b60bd60a259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:08.430831 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e6585-5420-4d9e-8888-1b60bd60a259 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:08.431118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e6585-5420-4d9e-8888-1b60bd60a259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:08.431470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e6585-5420-4d9e-8888-1b60bd60a259 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:08.432085 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190e6585-5420-4d9e-8888-1b60bd60a259 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:08.434030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190e6585-5420-4d9e-8888-1b60bd60a259 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:08.462177 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1127/2272] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:31:08 2022] POST /identity/v3/projects => generated 359 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:31:08.466143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1d253-4472-4f6f-87af-99c1b014c4e6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:08.477661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1d253-4472-4f6f-87af-99c1b014c4e6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:08.536781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c09cd-9b45-4ab6-8138-cecc55e5a6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:08.537477 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c09cd-9b45-4ab6-8138-cecc55e5a6b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:08.537861 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c09cd-9b45-4ab6-8138-cecc55e5a6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:08.538141 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c09cd-9b45-4ab6-8138-cecc55e5a6b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:08.538829 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636c09cd-9b45-4ab6-8138-cecc55e5a6b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:08.540747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636c09cd-9b45-4ab6-8138-cecc55e5a6b5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:08.560571 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1146/2273] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:31:08 2022] POST /identity/v3/users => generated 280 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:31:08.564544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e6585-5420-4d9e-8888-1b60bd60a259 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:08.575645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e6585-5420-4d9e-8888-1b60bd60a259 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:08.584303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65afce9-a2a9-4964-a8d0-2b527b4ab9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:08.584966 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65afce9-a2a9-4964-a8d0-2b527b4ab9b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:08.585254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65afce9-a2a9-4964-a8d0-2b527b4ab9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:08.585536 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65afce9-a2a9-4964-a8d0-2b527b4ab9b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:08.586298 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65afce9-a2a9-4964-a8d0-2b527b4ab9b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:08.587937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65afce9-a2a9-4964-a8d0-2b527b4ab9b5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:08.600339 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1128/2274] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:31:08.604431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c09cd-9b45-4ab6-8138-cecc55e5a6b5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:08.616042 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c09cd-9b45-4ab6-8138-cecc55e5a6b5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:08.621231 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3a817-d318-452e-bd0a-2012681ff3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:08.621938 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3a817-d318-452e-bd0a-2012681ff3d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:08.622227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3a817-d318-452e-bd0a-2012681ff3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:08.622508 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3a817-d318-452e-bd0a-2012681ff3d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:08.623238 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3a817-d318-452e-bd0a-2012681ff3d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:08.624871 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3a817-d318-452e-bd0a-2012681ff3d1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:08.635039 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1147/2275] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:31:08.639168 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65afce9-a2a9-4964-a8d0-2b527b4ab9b5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:08.649926 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65afce9-a2a9-4964-a8d0-2b527b4ab9b5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:08.654510 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b346196-91fc-4faa-8969-079906ba0a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:08.656422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b346196-91fc-4faa-8969-079906ba0a8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:08.656654 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b346196-91fc-4faa-8969-079906ba0a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:08.656936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b346196-91fc-4faa-8969-079906ba0a8b admin admin] PATH_INFO: `/v3/projects/b974372994464b028df6c4dc6967c237/users/6758564d748d477b852a20162fce8365/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:08.673891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b346196-91fc-4faa-8969-079906ba0a8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b974372994464b028df6c4dc6967c237, user_id=6758564d748d477b852a20162fce8365, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:08.676576 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b346196-91fc-4faa-8969-079906ba0a8b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:08.689967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1129/2276] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:31:08 2022] PUT /identity/v3/projects/b974372994464b028df6c4dc6967c237/users/6758564d748d477b852a20162fce8365/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:31:08.701198 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b64d8-07a2-4f1d-9732-34c926414636 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:08.702015 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b64d8-07a2-4f1d-9732-34c926414636 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:08.702482 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b64d8-07a2-4f1d-9732-34c926414636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:08.722802 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:08.732330 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-423b64d8-07a2-4f1d-9732-34c926414636 None None] MFA Rules not processed for user `6758564d748d477b852a20162fce8365`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:08.734789 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-423b64d8-07a2-4f1d-9732-34c926414636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:08.798975 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1148/2277] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:31:08 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:08.829707 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b346196-91fc-4faa-8969-079906ba0a8b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:08.831385 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5b346196-91fc-4faa-8969-079906ba0a8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:08.886824 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b346196-91fc-4faa-8969-079906ba0a8b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:08.891042 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738a400-9f86-4d7a-9195-598e753fce90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:08.891661 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738a400-9f86-4d7a-9195-598e753fce90 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:08.891941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738a400-9f86-4d7a-9195-598e753fce90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:08.892213 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738a400-9f86-4d7a-9195-598e753fce90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:08.900281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738a400-9f86-4d7a-9195-598e753fce90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:08.901737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738a400-9f86-4d7a-9195-598e753fce90 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:08.910579 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1130/2278] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:31:08 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:31:08.924979 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b64d8-07a2-4f1d-9732-34c926414636 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:08.991085 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b64d8-07a2-4f1d-9732-34c926414636 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:08.995651 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32edab44-4b6c-4ebd-86bd-4d369c78a589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:08.996522 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32edab44-4b6c-4ebd-86bd-4d369c78a589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:08.996809 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32edab44-4b6c-4ebd-86bd-4d369c78a589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:08.997090 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32edab44-4b6c-4ebd-86bd-4d369c78a589 admin admin] PATH_INFO: `/v3/users/6758564d748d477b852a20162fce8365` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:08.998575 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32edab44-4b6c-4ebd-86bd-4d369c78a589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6758564d748d477b852a20162fce8365)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:09.001150 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32edab44-4b6c-4ebd-86bd-4d369c78a589 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:09.049896 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-32edab44-4b6c-4ebd-86bd-4d369c78a589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6758564d748d477b852a20162fce8365'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:09.061394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-32edab44-4b6c-4ebd-86bd-4d369c78a589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6758564d748d477b852a20162fce8365'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:09.062754 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-32edab44-4b6c-4ebd-86bd-4d369c78a589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6758564d748d477b852a20162fce8365'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:09.070187 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-32edab44-4b6c-4ebd-86bd-4d369c78a589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6758564d748d477b852a20162fce8365'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:09.070869 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-32edab44-4b6c-4ebd-86bd-4d369c78a589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6758564d748d477b852a20162fce8365'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:09.083248 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1149/2279] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:31:08 2022] DELETE /identity/v3/users/6758564d748d477b852a20162fce8365 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:31:09.087822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dddd95-712c-4871-9645-a043ee0530d7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:09.088114 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dddd95-712c-4871-9645-a043ee0530d7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:09.088396 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dddd95-712c-4871-9645-a043ee0530d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:09.105091 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-88dddd95-712c-4871-9645-a043ee0530d7 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:09.106338 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-88dddd95-712c-4871-9645-a043ee0530d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:09.119506 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:09.174817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1131/2280] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:31:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:09.184275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32edab44-4b6c-4ebd-86bd-4d369c78a589 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:09.185709 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-32edab44-4b6c-4ebd-86bd-4d369c78a589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:09.206690 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:09.244938 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32edab44-4b6c-4ebd-86bd-4d369c78a589 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:09.248682 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ee453-176f-45d9-a01e-2e7d65971203 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:09.250051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ee453-176f-45d9-a01e-2e7d65971203 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:09.250347 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ee453-176f-45d9-a01e-2e7d65971203 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:09.250636 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ee453-176f-45d9-a01e-2e7d65971203 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:09.259744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ee453-176f-45d9-a01e-2e7d65971203 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:09.261221 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ee453-176f-45d9-a01e-2e7d65971203 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:09.271287 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1150/2281] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:31:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:31:09.562550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dddd95-712c-4871-9645-a043ee0530d7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:09.564027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-88dddd95-712c-4871-9645-a043ee0530d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:09.577612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dddd95-712c-4871-9645-a043ee0530d7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:09.584276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d481ddb-e943-436c-bf70-0f4a6664c891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:09.585404 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d481ddb-e943-436c-bf70-0f4a6664c891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:09.586291 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d481ddb-e943-436c-bf70-0f4a6664c891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:09.586692 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d481ddb-e943-436c-bf70-0f4a6664c891 admin admin] PATH_INFO: `/v3/projects/b974372994464b028df6c4dc6967c237` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:09.588063 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d481ddb-e943-436c-bf70-0f4a6664c891 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b974372994464b028df6c4dc6967c237)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:09.590725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d481ddb-e943-436c-bf70-0f4a6664c891 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:09.656202 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8d481ddb-e943-436c-bf70-0f4a6664c891 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b974372994464b028df6c4dc6967c237', 'request_id': 'req-8d481ddb-e943-436c-bf70-0f4a6664c891'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:09.662835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8d481ddb-e943-436c-bf70-0f4a6664c891 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b974372994464b028df6c4dc6967c237', 'request_id': 'req-8d481ddb-e943-436c-bf70-0f4a6664c891'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:09.668579 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8d481ddb-e943-436c-bf70-0f4a6664c891 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b974372994464b028df6c4dc6967c237', 'request_id': 'req-8d481ddb-e943-436c-bf70-0f4a6664c891'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:09.681355 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8d481ddb-e943-436c-bf70-0f4a6664c891 admin admin] The token cache is being invalidate because project b974372994464b028df6c4dc6967c237 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:31:09.681561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8d481ddb-e943-436c-bf70-0f4a6664c891 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:09.682235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8d481ddb-e943-436c-bf70-0f4a6664c891 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:09.683808 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1132/2282] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:31:09 2022] DELETE /identity/v3/projects/b974372994464b028df6c4dc6967c237 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:31:09.689778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d47873-b76f-4398-8b63-27b7b9501d3c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:09.690288 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d47873-b76f-4398-8b63-27b7b9501d3c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:09.690405 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d47873-b76f-4398-8b63-27b7b9501d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:09.709039 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-60d47873-b76f-4398-8b63-27b7b9501d3c None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:09.710319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-60d47873-b76f-4398-8b63-27b7b9501d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:09.796203 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1151/2283] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:31:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:31:09.804816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65e24e-0540-45ed-857c-a4dd344c045e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:09.805228 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65e24e-0540-45ed-857c-a4dd344c045e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:09.805644 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65e24e-0540-45ed-857c-a4dd344c045e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:09.825172 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8b65e24e-0540-45ed-857c-a4dd344c045e None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:09.826429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8b65e24e-0540-45ed-857c-a4dd344c045e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:09.835508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1133/2284] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:31:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:09.839691 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d47873-b76f-4398-8b63-27b7b9501d3c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:09.850767 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d47873-b76f-4398-8b63-27b7b9501d3c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:09.855513 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3afed-c7d6-45a3-be32-741c95fecd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:09.856170 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3afed-c7d6-45a3-be32-741c95fecd71 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:09.856457 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3afed-c7d6-45a3-be32-741c95fecd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:09.856740 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3afed-c7d6-45a3-be32-741c95fecd71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:09.857470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a3afed-c7d6-45a3-be32-741c95fecd71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:31:09.857815 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a3afed-c7d6-45a3-be32-741c95fecd71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:09.859670 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a3afed-c7d6-45a3-be32-741c95fecd71 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:09.877123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1152/2285] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:31:09.878262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65e24e-0540-45ed-857c-a4dd344c045e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:09.888053 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65e24e-0540-45ed-857c-a4dd344c045e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:09.893821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111508d9-031f-4116-9106-4f3803358c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:09.894508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111508d9-031f-4116-9106-4f3803358c80 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:09.894862 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111508d9-031f-4116-9106-4f3803358c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:09.895061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111508d9-031f-4116-9106-4f3803358c80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:09.895770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111508d9-031f-4116-9106-4f3803358c80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:09.897790 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111508d9-031f-4116-9106-4f3803358c80 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:09.930323 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1134/2286] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:31:09 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:31:09.931511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3afed-c7d6-45a3-be32-741c95fecd71 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:09.942710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3afed-c7d6-45a3-be32-741c95fecd71 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:10.002070 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca5870-d20b-4181-ad44-75897f10cc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:10.002747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca5870-d20b-4181-ad44-75897f10cc97 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:10.003036 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca5870-d20b-4181-ad44-75897f10cc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:10.003320 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca5870-d20b-4181-ad44-75897f10cc97 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:10.004761 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca5870-d20b-4181-ad44-75897f10cc97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:10.005993 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca5870-d20b-4181-ad44-75897f10cc97 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:10.029199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1153/2287] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:31:09 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:31:10.033909 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111508d9-031f-4116-9106-4f3803358c80 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:10.045342 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111508d9-031f-4116-9106-4f3803358c80 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:10.050563 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832f8b1-7997-4c3e-a503-81ef501e5884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:10.051261 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832f8b1-7997-4c3e-a503-81ef501e5884 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:10.051547 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832f8b1-7997-4c3e-a503-81ef501e5884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:10.051828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832f8b1-7997-4c3e-a503-81ef501e5884 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:10.052596 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5832f8b1-7997-4c3e-a503-81ef501e5884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:10.054284 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5832f8b1-7997-4c3e-a503-81ef501e5884 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:10.066945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1135/2288] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:31:10.070890 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca5870-d20b-4181-ad44-75897f10cc97 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:10.082049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca5870-d20b-4181-ad44-75897f10cc97 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:10.086796 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ac390-9b48-46aa-9a4e-2ef303409c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:10.087464 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ac390-9b48-46aa-9a4e-2ef303409c3a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:10.087754 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ac390-9b48-46aa-9a4e-2ef303409c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:10.088244 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ac390-9b48-46aa-9a4e-2ef303409c3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:10.088986 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ac390-9b48-46aa-9a4e-2ef303409c3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:10.090638 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ac390-9b48-46aa-9a4e-2ef303409c3a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:10.100917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1154/2289] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:31:10.105319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832f8b1-7997-4c3e-a503-81ef501e5884 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:10.115479 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832f8b1-7997-4c3e-a503-81ef501e5884 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:10.120193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e3c9b-67d1-42f0-ad62-b42816f3e9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:10.121070 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e3c9b-67d1-42f0-ad62-b42816f3e9ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:10.121354 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e3c9b-67d1-42f0-ad62-b42816f3e9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:10.121673 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e3c9b-67d1-42f0-ad62-b42816f3e9ce admin admin] PATH_INFO: `/v3/projects/ce15595bde2f479ea9e08015d30660c5/users/f59dcd1d5ad44ce28ba11ff5ba5df534/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:10.138351 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e3c9b-67d1-42f0-ad62-b42816f3e9ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce15595bde2f479ea9e08015d30660c5, user_id=f59dcd1d5ad44ce28ba11ff5ba5df534, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:10.142098 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e3c9b-67d1-42f0-ad62-b42816f3e9ce admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:10.156397 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1136/2290] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:31:10 2022] PUT /identity/v3/projects/ce15595bde2f479ea9e08015d30660c5/users/f59dcd1d5ad44ce28ba11ff5ba5df534/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:31:10.167661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141434cd-1584-4a25-9d99-f24e6cc7afff None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:10.167956 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141434cd-1584-4a25-9d99-f24e6cc7afff None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:10.168262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141434cd-1584-4a25-9d99-f24e6cc7afff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:10.195978 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-141434cd-1584-4a25-9d99-f24e6cc7afff None None] MFA Rules not processed for user `f59dcd1d5ad44ce28ba11ff5ba5df534`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:10.197127 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-141434cd-1584-4a25-9d99-f24e6cc7afff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:10.212407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:10.266385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1155/2291] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:31:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:10.270940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e3c9b-67d1-42f0-ad62-b42816f3e9ce admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:10.319010 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:10.342427 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e3c9b-67d1-42f0-ad62-b42816f3e9ce admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:10.348178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f7535-88d8-46b3-b96e-2b5d40e7f1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:10.348895 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f7535-88d8-46b3-b96e-2b5d40e7f1bf admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:10.349190 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f7535-88d8-46b3-b96e-2b5d40e7f1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:10.349462 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f7535-88d8-46b3-b96e-2b5d40e7f1bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:10.350256 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f7535-88d8-46b3-b96e-2b5d40e7f1bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:10.352300 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f7535-88d8-46b3-b96e-2b5d40e7f1bf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:10.381815 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1137/2292] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:31:10 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:31:10.383451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141434cd-1584-4a25-9d99-f24e6cc7afff None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:10.395353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141434cd-1584-4a25-9d99-f24e6cc7afff None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:10.461994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fbc1f7-cf54-4937-945b-b3985e51248b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:10.462717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbc1f7-cf54-4937-945b-b3985e51248b admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:10.462994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbc1f7-cf54-4937-945b-b3985e51248b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:10.463284 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbc1f7-cf54-4937-945b-b3985e51248b admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:10.464027 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fbc1f7-cf54-4937-945b-b3985e51248b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:10.466269 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fbc1f7-cf54-4937-945b-b3985e51248b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:10.489619 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1156/2293] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:31:10 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:31:10.491389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f7535-88d8-46b3-b96e-2b5d40e7f1bf admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:10.503137 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f7535-88d8-46b3-b96e-2b5d40e7f1bf admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:10.508901 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1e9b9-0f0a-4201-aa3b-76cccfc1891c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:10.509566 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1e9b9-0f0a-4201-aa3b-76cccfc1891c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:10.509880 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1e9b9-0f0a-4201-aa3b-76cccfc1891c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:10.510162 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1e9b9-0f0a-4201-aa3b-76cccfc1891c admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:10.510895 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1e9b9-0f0a-4201-aa3b-76cccfc1891c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:10.512510 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1e9b9-0f0a-4201-aa3b-76cccfc1891c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:10.524023 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1138/2294] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:31:10.528197 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fbc1f7-cf54-4937-945b-b3985e51248b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:10.538565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fbc1f7-cf54-4937-945b-b3985e51248b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:10.543962 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f58692-57a8-4691-a661-d8a93cc79bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:10.544629 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f58692-57a8-4691-a661-d8a93cc79bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:10.544919 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f58692-57a8-4691-a661-d8a93cc79bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:10.545210 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f58692-57a8-4691-a661-d8a93cc79bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:10.546534 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f58692-57a8-4691-a661-d8a93cc79bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:10.548380 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f58692-57a8-4691-a661-d8a93cc79bd4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:10.558578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1157/2295] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:31:10.563616 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1e9b9-0f0a-4201-aa3b-76cccfc1891c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:10.574486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1e9b9-0f0a-4201-aa3b-76cccfc1891c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:10.580651 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ddb24-aa46-4d3b-92d6-a17eeb2600d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:10.581546 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ddb24-aa46-4d3b-92d6-a17eeb2600d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:10.582318 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ddb24-aa46-4d3b-92d6-a17eeb2600d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:10.582611 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ddb24-aa46-4d3b-92d6-a17eeb2600d8 admin admin] PATH_INFO: `/v3/projects/b6565312c47744b9995ac176c01df397/users/aca3666e04c4419dbc646c8ff72b0cea/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:10.599502 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ddb24-aa46-4d3b-92d6-a17eeb2600d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6565312c47744b9995ac176c01df397, user_id=aca3666e04c4419dbc646c8ff72b0cea, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:10.602234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ddb24-aa46-4d3b-92d6-a17eeb2600d8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:10.618542 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1139/2296] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:31:10 2022] PUT /identity/v3/projects/b6565312c47744b9995ac176c01df397/users/aca3666e04c4419dbc646c8ff72b0cea/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:31:10.629723 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d0b41-bcaf-4767-8d76-887a90c22746 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:10.629723 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d0b41-bcaf-4767-8d76-887a90c22746 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:10.630043 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d0b41-bcaf-4767-8d76-887a90c22746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:10.663988 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0f8d0b41-bcaf-4767-8d76-887a90c22746 None None] MFA Rules not processed for user `aca3666e04c4419dbc646c8ff72b0cea`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:10.663988 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0f8d0b41-bcaf-4767-8d76-887a90c22746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:10.665299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba3206-37c2-43f6-b4c3-9f925fd9867c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:10.665767 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba3206-37c2-43f6-b4c3-9f925fd9867c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:10.665893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba3206-37c2-43f6-b4c3-9f925fd9867c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:10.685433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ffba3206-37c2-43f6-b4c3-9f925fd9867c None None] MFA Rules not processed for user `c66efdb1cc27467d85831617590c8b28`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:10.687691 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ffba3206-37c2-43f6-b4c3-9f925fd9867c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:10.752565 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1140/2297] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:31:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:10.756893 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1158/2298] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:31:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:10.761286 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba3206-37c2-43f6-b4c3-9f925fd9867c None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:10.834748 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba3206-37c2-43f6-b4c3-9f925fd9867c None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:10.840926 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfcc375-ddde-4067-9179-c5660dffc427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:10.841735 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcc375-ddde-4067-9179-c5660dffc427 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:10.842127 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcc375-ddde-4067-9179-c5660dffc427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:10.842504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcc375-ddde-4067-9179-c5660dffc427 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:10.843878 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfcc375-ddde-4067-9179-c5660dffc427 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:10.845955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfcc375-ddde-4067-9179-c5660dffc427 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:10.873171 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1141/2299] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:31:10 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:31:10.878380 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d0b41-bcaf-4767-8d76-887a90c22746 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:10.889428 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d0b41-bcaf-4767-8d76-887a90c22746 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:10.910764 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:10.955447 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792f953-ed35-4f4f-a63e-acf26df668de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:10.956133 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792f953-ed35-4f4f-a63e-acf26df668de admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:10.956419 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792f953-ed35-4f4f-a63e-acf26df668de admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:10.956730 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792f953-ed35-4f4f-a63e-acf26df668de admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:10.957482 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792f953-ed35-4f4f-a63e-acf26df668de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:10.959470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792f953-ed35-4f4f-a63e-acf26df668de admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:10.984276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1159/2300] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:31:10 2022] POST /identity/v3/users => generated 295 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:31:10.988441 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfcc375-ddde-4067-9179-c5660dffc427 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:10.998901 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfcc375-ddde-4067-9179-c5660dffc427 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:11.004172 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490638fc-fdb5-4489-aecd-24ac7bfb3ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:11.004839 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490638fc-fdb5-4489-aecd-24ac7bfb3ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:11.005127 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490638fc-fdb5-4489-aecd-24ac7bfb3ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:11.005423 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490638fc-fdb5-4489-aecd-24ac7bfb3ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:11.006183 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490638fc-fdb5-4489-aecd-24ac7bfb3ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:11.007793 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490638fc-fdb5-4489-aecd-24ac7bfb3ba9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:11.019336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1142/2301] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:31:11.023680 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792f953-ed35-4f4f-a63e-acf26df668de admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:11.034836 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792f953-ed35-4f4f-a63e-acf26df668de admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:11.039618 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8772eb-6ef5-4c1a-80c3-53891f1702c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:11.040561 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8772eb-6ef5-4c1a-80c3-53891f1702c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:11.040826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8772eb-6ef5-4c1a-80c3-53891f1702c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:11.041113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8772eb-6ef5-4c1a-80c3-53891f1702c1 admin admin] PATH_INFO: `/v3/projects/20e7f17ccb364b5293e97fa8e4ede0f3/users/70addaf8b96a4818bc7803501199cd45/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:11.058126 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8772eb-6ef5-4c1a-80c3-53891f1702c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20e7f17ccb364b5293e97fa8e4ede0f3, user_id=70addaf8b96a4818bc7803501199cd45, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:11.060830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8772eb-6ef5-4c1a-80c3-53891f1702c1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:11.080319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1160/2302] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:31:11 2022] PUT /identity/v3/projects/20e7f17ccb364b5293e97fa8e4ede0f3/users/70addaf8b96a4818bc7803501199cd45/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:31:11.084376 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490638fc-fdb5-4489-aecd-24ac7bfb3ba9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:11.095123 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490638fc-fdb5-4489-aecd-24ac7bfb3ba9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:11.100032 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:11.151432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c85082-1078-4723-b064-ca0f8198e611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:11.152192 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c85082-1078-4723-b064-ca0f8198e611 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:11.152548 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c85082-1078-4723-b064-ca0f8198e611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:11.152821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c85082-1078-4723-b064-ca0f8198e611 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:11.153541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c85082-1078-4723-b064-ca0f8198e611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:11.155164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c85082-1078-4723-b064-ca0f8198e611 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:11.169037 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1143/2303] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:31:11.170461 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8772eb-6ef5-4c1a-80c3-53891f1702c1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:11.182266 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8772eb-6ef5-4c1a-80c3-53891f1702c1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:11.187064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fe80f-bb24-48fd-926b-8980ffc54b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:11.187954 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fe80f-bb24-48fd-926b-8980ffc54b1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:11.188245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fe80f-bb24-48fd-926b-8980ffc54b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:11.188527 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fe80f-bb24-48fd-926b-8980ffc54b1f admin admin] PATH_INFO: `/v3/projects/20e7f17ccb364b5293e97fa8e4ede0f3/users/70addaf8b96a4818bc7803501199cd45/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:11.190580 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fe80f-bb24-48fd-926b-8980ffc54b1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=20e7f17ccb364b5293e97fa8e4ede0f3, user_id=70addaf8b96a4818bc7803501199cd45, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:11.193500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fe80f-bb24-48fd-926b-8980ffc54b1f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:11.205860 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1161/2304] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:31:11 2022] PUT /identity/v3/projects/20e7f17ccb364b5293e97fa8e4ede0f3/users/70addaf8b96a4818bc7803501199cd45/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:31:11.216071 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6833a-e9f3-41da-b299-52860e151f41 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:11.216361 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6833a-e9f3-41da-b299-52860e151f41 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:11.216642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6833a-e9f3-41da-b299-52860e151f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:11.244447 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-94d6833a-e9f3-41da-b299-52860e151f41 None None] MFA Rules not processed for user `70addaf8b96a4818bc7803501199cd45`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:11.245606 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-94d6833a-e9f3-41da-b299-52860e151f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:11.318139 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1144/2305] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:31:11 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:31:11.330126 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fe80f-bb24-48fd-926b-8980ffc54b1f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:11.332107 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6c9fe80f-bb24-48fd-926b-8980ffc54b1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:11.412177 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fe80f-bb24-48fd-926b-8980ffc54b1f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:11.415890 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f72ce-d770-4e98-b807-36ed563d1497 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:11.416550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f72ce-d770-4e98-b807-36ed563d1497 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:11.416973 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f72ce-d770-4e98-b807-36ed563d1497 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:11.417260 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f72ce-d770-4e98-b807-36ed563d1497 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:11.484953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f72ce-d770-4e98-b807-36ed563d1497 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:11.486431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f72ce-d770-4e98-b807-36ed563d1497 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:11.495437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1162/2306] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:31:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:31:14.059460 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8132ac-2f6c-4a87-b563-967f4ecd2a85 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:14.059898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8132ac-2f6c-4a87-b563-967f4ecd2a85 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:14.059898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8132ac-2f6c-4a87-b563-967f4ecd2a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:14.073283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f72ce-d770-4e98-b807-36ed563d1497 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:14.074752 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-be7f72ce-d770-4e98-b807-36ed563d1497 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:14.078296 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-0c8132ac-2f6c-4a87-b563-967f4ecd2a85 None None] MFA Rules not processed for user `374555cda1b4424894093ebcc51cd651`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:14.079489 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0c8132ac-2f6c-4a87-b563-967f4ecd2a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:14.086711 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f72ce-d770-4e98-b807-36ed563d1497 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:14.090380 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4003b-c00a-4395-a20c-e054335ae5c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:14.091017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4003b-c00a-4395-a20c-e054335ae5c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:14.091305 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4003b-c00a-4395-a20c-e054335ae5c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:14.091585 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4003b-c00a-4395-a20c-e054335ae5c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:14.128196 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:14.148611 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1145/2307] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:31:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:14.162568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d4003b-c00a-4395-a20c-e054335ae5c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:14.164018 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d4003b-c00a-4395-a20c-e054335ae5c6 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:14.174674 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1163/2308] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:31:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:31:14.821112 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8132ac-2f6c-4a87-b563-967f4ecd2a85 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:14.834838 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8132ac-2f6c-4a87-b563-967f4ecd2a85 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:14.840361 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8711c87-98ad-4d90-9768-6c6ba41f1f0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:14.841604 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8711c87-98ad-4d90-9768-6c6ba41f1f0b service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:14.842276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8711c87-98ad-4d90-9768-6c6ba41f1f0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:14.843273 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8711c87-98ad-4d90-9768-6c6ba41f1f0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:14.858360 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8711c87-98ad-4d90-9768-6c6ba41f1f0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:14.860538 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8711c87-98ad-4d90-9768-6c6ba41f1f0b service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:14.896959 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1146/2309] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:31:14 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:31:24.168842 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=14 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:24.876602 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:27.314303 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4003b-c00a-4395-a20c-e054335ae5c6 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:27.378155 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4003b-c00a-4395-a20c-e054335ae5c6 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:27.382734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5234a8-16d9-4df1-9e7e-c488cd3cd123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:27.383594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5234a8-16d9-4df1-9e7e-c488cd3cd123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:27.383883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5234a8-16d9-4df1-9e7e-c488cd3cd123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:27.384168 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5234a8-16d9-4df1-9e7e-c488cd3cd123 admin admin] PATH_INFO: `/v3/users/f59dcd1d5ad44ce28ba11ff5ba5df534` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:27.385352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5234a8-16d9-4df1-9e7e-c488cd3cd123 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f59dcd1d5ad44ce28ba11ff5ba5df534)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:27.387320 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5234a8-16d9-4df1-9e7e-c488cd3cd123 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:27.432066 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1b5234a8-16d9-4df1-9e7e-c488cd3cd123 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f59dcd1d5ad44ce28ba11ff5ba5df534'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:27.444328 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1b5234a8-16d9-4df1-9e7e-c488cd3cd123 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f59dcd1d5ad44ce28ba11ff5ba5df534'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:27.444994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1b5234a8-16d9-4df1-9e7e-c488cd3cd123 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f59dcd1d5ad44ce28ba11ff5ba5df534'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:27.451809 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1b5234a8-16d9-4df1-9e7e-c488cd3cd123 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f59dcd1d5ad44ce28ba11ff5ba5df534'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:27.452437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-1b5234a8-16d9-4df1-9e7e-c488cd3cd123 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f59dcd1d5ad44ce28ba11ff5ba5df534'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:27.465415 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1164/2310] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:31:27 2022] DELETE /identity/v3/users/f59dcd1d5ad44ce28ba11ff5ba5df534 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:31:27.470605 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77413a5b-6020-4fe6-bd75-620c3aefb6bd None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:27.471075 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77413a5b-6020-4fe6-bd75-620c3aefb6bd None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:27.471170 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77413a5b-6020-4fe6-bd75-620c3aefb6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:27.493707 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-77413a5b-6020-4fe6-bd75-620c3aefb6bd None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:27.495004 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-77413a5b-6020-4fe6-bd75-620c3aefb6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:27.554061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1147/2311] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:31:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:27.563959 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5234a8-16d9-4df1-9e7e-c488cd3cd123 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:27.565377 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1b5234a8-16d9-4df1-9e7e-c488cd3cd123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:27.628638 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5234a8-16d9-4df1-9e7e-c488cd3cd123 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:27.632375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca8661-1b40-415c-a5e7-b360aee414fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:27.633024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca8661-1b40-415c-a5e7-b360aee414fc service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:27.633600 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca8661-1b40-415c-a5e7-b360aee414fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:27.633896 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca8661-1b40-415c-a5e7-b360aee414fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:27.644097 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca8661-1b40-415c-a5e7-b360aee414fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:27.645536 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca8661-1b40-415c-a5e7-b360aee414fc service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:27.655275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1165/2312] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:31:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:31:27.962874 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77413a5b-6020-4fe6-bd75-620c3aefb6bd None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:27.964437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-77413a5b-6020-4fe6-bd75-620c3aefb6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:27.977071 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77413a5b-6020-4fe6-bd75-620c3aefb6bd None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:27.981755 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acd098-958e-4530-8273-908a9d9f021a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:27.982619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acd098-958e-4530-8273-908a9d9f021a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:27.982912 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acd098-958e-4530-8273-908a9d9f021a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:27.983188 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acd098-958e-4530-8273-908a9d9f021a admin admin] PATH_INFO: `/v3/projects/ce15595bde2f479ea9e08015d30660c5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:27.985157 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2acd098-958e-4530-8273-908a9d9f021a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce15595bde2f479ea9e08015d30660c5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:27.987604 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2acd098-958e-4530-8273-908a9d9f021a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:28.034983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e2acd098-958e-4530-8273-908a9d9f021a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce15595bde2f479ea9e08015d30660c5', 'request_id': 'req-e2acd098-958e-4530-8273-908a9d9f021a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:28.041066 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e2acd098-958e-4530-8273-908a9d9f021a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce15595bde2f479ea9e08015d30660c5', 'request_id': 'req-e2acd098-958e-4530-8273-908a9d9f021a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:28.046611 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e2acd098-958e-4530-8273-908a9d9f021a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce15595bde2f479ea9e08015d30660c5', 'request_id': 'req-e2acd098-958e-4530-8273-908a9d9f021a'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:28.059004 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e2acd098-958e-4530-8273-908a9d9f021a admin admin] The token cache is being invalidate because project ce15595bde2f479ea9e08015d30660c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:31:28.059296 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e2acd098-958e-4530-8273-908a9d9f021a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:28.059919 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e2acd098-958e-4530-8273-908a9d9f021a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:28.061548 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1148/2313] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:31:27 2022] DELETE /identity/v3/projects/ce15595bde2f479ea9e08015d30660c5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:31:28.066044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca8661-1b40-415c-a5e7-b360aee414fc service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:28.075929 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca8661-1b40-415c-a5e7-b360aee414fc service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:28.094879 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:28.135526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c5253-65e7-4c4f-a910-2f8f1e9b37f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:28.136398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c5253-65e7-4c4f-a910-2f8f1e9b37f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:28.136695 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c5253-65e7-4c4f-a910-2f8f1e9b37f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:28.136982 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c5253-65e7-4c4f-a910-2f8f1e9b37f8 admin admin] PATH_INFO: `/v3/users/aca3666e04c4419dbc646c8ff72b0cea` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:28.138197 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c5253-65e7-4c4f-a910-2f8f1e9b37f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aca3666e04c4419dbc646c8ff72b0cea)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:28.140126 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c5253-65e7-4c4f-a910-2f8f1e9b37f8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:28.184540 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-207c5253-65e7-4c4f-a910-2f8f1e9b37f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aca3666e04c4419dbc646c8ff72b0cea'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:28.197538 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-207c5253-65e7-4c4f-a910-2f8f1e9b37f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aca3666e04c4419dbc646c8ff72b0cea'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:28.198274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-207c5253-65e7-4c4f-a910-2f8f1e9b37f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aca3666e04c4419dbc646c8ff72b0cea'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:28.205922 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-207c5253-65e7-4c4f-a910-2f8f1e9b37f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aca3666e04c4419dbc646c8ff72b0cea'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:28.206557 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-207c5253-65e7-4c4f-a910-2f8f1e9b37f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aca3666e04c4419dbc646c8ff72b0cea'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:28.218799 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1166/2314] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:31:28 2022] DELETE /identity/v3/users/aca3666e04c4419dbc646c8ff72b0cea => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:31:28.486561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acd098-958e-4530-8273-908a9d9f021a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:28.488066 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e2acd098-958e-4530-8273-908a9d9f021a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:28.510045 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acd098-958e-4530-8273-908a9d9f021a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:28.568896 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87abdf2-3dbd-4bfe-8fa4-c96e6479f7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:28.569706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87abdf2-3dbd-4bfe-8fa4-c96e6479f7f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:28.569990 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87abdf2-3dbd-4bfe-8fa4-c96e6479f7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:28.570271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87abdf2-3dbd-4bfe-8fa4-c96e6479f7f6 admin admin] PATH_INFO: `/v3/projects/b6565312c47744b9995ac176c01df397` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:28.571452 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87abdf2-3dbd-4bfe-8fa4-c96e6479f7f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6565312c47744b9995ac176c01df397)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:28.573372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87abdf2-3dbd-4bfe-8fa4-c96e6479f7f6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:28.604786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:28.622892 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c87abdf2-3dbd-4bfe-8fa4-c96e6479f7f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6565312c47744b9995ac176c01df397', 'request_id': 'req-c87abdf2-3dbd-4bfe-8fa4-c96e6479f7f6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:28.629204 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c87abdf2-3dbd-4bfe-8fa4-c96e6479f7f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6565312c47744b9995ac176c01df397', 'request_id': 'req-c87abdf2-3dbd-4bfe-8fa4-c96e6479f7f6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:28.634650 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c87abdf2-3dbd-4bfe-8fa4-c96e6479f7f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6565312c47744b9995ac176c01df397', 'request_id': 'req-c87abdf2-3dbd-4bfe-8fa4-c96e6479f7f6'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:28.647425 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c87abdf2-3dbd-4bfe-8fa4-c96e6479f7f6 admin admin] The token cache is being invalidate because project b6565312c47744b9995ac176c01df397 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:31:28.647713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c87abdf2-3dbd-4bfe-8fa4-c96e6479f7f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:28.648695 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c87abdf2-3dbd-4bfe-8fa4-c96e6479f7f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:28.649947 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1149/2315] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:31:28 2022] DELETE /identity/v3/projects/b6565312c47744b9995ac176c01df397 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:31:28.653964 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c5253-65e7-4c4f-a910-2f8f1e9b37f8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:28.655485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-207c5253-65e7-4c4f-a910-2f8f1e9b37f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:28.670349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c5253-65e7-4c4f-a910-2f8f1e9b37f8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:28.733327 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0195c7-5311-457e-b411-5a07a3dfde4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:28.734236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0195c7-5311-457e-b411-5a07a3dfde4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:28.734692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0195c7-5311-457e-b411-5a07a3dfde4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:28.735187 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0195c7-5311-457e-b411-5a07a3dfde4e admin admin] PATH_INFO: `/v3/users/70addaf8b96a4818bc7803501199cd45` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:28.736390 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0195c7-5311-457e-b411-5a07a3dfde4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=70addaf8b96a4818bc7803501199cd45)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:28.738358 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0195c7-5311-457e-b411-5a07a3dfde4e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:28.785969 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:28.788945 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7a0195c7-5311-457e-b411-5a07a3dfde4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70addaf8b96a4818bc7803501199cd45'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:28.802888 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7a0195c7-5311-457e-b411-5a07a3dfde4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70addaf8b96a4818bc7803501199cd45'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:28.803628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7a0195c7-5311-457e-b411-5a07a3dfde4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70addaf8b96a4818bc7803501199cd45'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:28.817914 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7a0195c7-5311-457e-b411-5a07a3dfde4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70addaf8b96a4818bc7803501199cd45'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:28.818883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7a0195c7-5311-457e-b411-5a07a3dfde4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70addaf8b96a4818bc7803501199cd45'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:28.834141 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1167/2316] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:31:28 2022] DELETE /identity/v3/users/70addaf8b96a4818bc7803501199cd45 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:31:29.163160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87abdf2-3dbd-4bfe-8fa4-c96e6479f7f6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:29.164770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c87abdf2-3dbd-4bfe-8fa4-c96e6479f7f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:29.179481 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87abdf2-3dbd-4bfe-8fa4-c96e6479f7f6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:29.238349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771b27b-5e02-4e06-b537-b4af0acf9794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:29.239193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771b27b-5e02-4e06-b537-b4af0acf9794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:29.239477 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771b27b-5e02-4e06-b537-b4af0acf9794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:29.239775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771b27b-5e02-4e06-b537-b4af0acf9794 admin admin] PATH_INFO: `/v3/projects/20e7f17ccb364b5293e97fa8e4ede0f3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:29.240922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771b27b-5e02-4e06-b537-b4af0acf9794 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20e7f17ccb364b5293e97fa8e4ede0f3)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:29.242871 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771b27b-5e02-4e06-b537-b4af0acf9794 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:29.295352 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6771b27b-5e02-4e06-b537-b4af0acf9794 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20e7f17ccb364b5293e97fa8e4ede0f3', 'request_id': 'req-6771b27b-5e02-4e06-b537-b4af0acf9794'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:29.302189 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6771b27b-5e02-4e06-b537-b4af0acf9794 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20e7f17ccb364b5293e97fa8e4ede0f3', 'request_id': 'req-6771b27b-5e02-4e06-b537-b4af0acf9794'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:29.307812 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6771b27b-5e02-4e06-b537-b4af0acf9794 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20e7f17ccb364b5293e97fa8e4ede0f3', 'request_id': 'req-6771b27b-5e02-4e06-b537-b4af0acf9794'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:29.319625 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6771b27b-5e02-4e06-b537-b4af0acf9794 admin admin] The token cache is being invalidate because project 20e7f17ccb364b5293e97fa8e4ede0f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:31:29.319909 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6771b27b-5e02-4e06-b537-b4af0acf9794 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:29.320636 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6771b27b-5e02-4e06-b537-b4af0acf9794 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:29.322198 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1150/2317] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:31:29 2022] DELETE /identity/v3/projects/20e7f17ccb364b5293e97fa8e4ede0f3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:31:29.327054 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08b3af-5926-47e6-874a-9ff818a676a7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:29.327349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08b3af-5926-47e6-874a-9ff818a676a7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:29.327634 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08b3af-5926-47e6-874a-9ff818a676a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:29.346702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7a08b3af-5926-47e6-874a-9ff818a676a7 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:29.347854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7a08b3af-5926-47e6-874a-9ff818a676a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:29.407012 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1168/2318] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:29.415971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37f225-30f6-4130-82c3-0f8576b83cb0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:29.416267 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37f225-30f6-4130-82c3-0f8576b83cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:29.416552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37f225-30f6-4130-82c3-0f8576b83cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:29.434735 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-af37f225-30f6-4130-82c3-0f8576b83cb0 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:29.435943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-af37f225-30f6-4130-82c3-0f8576b83cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:29.444260 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1151/2319] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:29.449173 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08b3af-5926-47e6-874a-9ff818a676a7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:29.459274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08b3af-5926-47e6-874a-9ff818a676a7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:29.463917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b154db-e113-4b28-8407-8f09d4132897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:29.464564 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b154db-e113-4b28-8407-8f09d4132897 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:29.464852 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b154db-e113-4b28-8407-8f09d4132897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:29.465135 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b154db-e113-4b28-8407-8f09d4132897 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:29.465908 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b154db-e113-4b28-8407-8f09d4132897 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:31:29.466216 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b154db-e113-4b28-8407-8f09d4132897 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:29.467823 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b154db-e113-4b28-8407-8f09d4132897 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:29.479785 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1169/2320] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:31:29.484165 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37f225-30f6-4130-82c3-0f8576b83cb0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:29.493505 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37f225-30f6-4130-82c3-0f8576b83cb0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:29.498113 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df83b8-ace9-43ad-9994-8b970dee4d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:29.498771 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df83b8-ace9-43ad-9994-8b970dee4d1b admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:29.499050 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df83b8-ace9-43ad-9994-8b970dee4d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:29.499350 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df83b8-ace9-43ad-9994-8b970dee4d1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:29.500919 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df83b8-ace9-43ad-9994-8b970dee4d1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:29.503371 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df83b8-ace9-43ad-9994-8b970dee4d1b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:29.530783 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1152/2321] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:31:29 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:31:29.535042 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b154db-e113-4b28-8407-8f09d4132897 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:29.545779 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b154db-e113-4b28-8407-8f09d4132897 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:29.604211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacaa5d2-180c-412c-80b0-041bf856cefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:29.604794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacaa5d2-180c-412c-80b0-041bf856cefa admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:29.605079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacaa5d2-180c-412c-80b0-041bf856cefa admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:29.605357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacaa5d2-180c-412c-80b0-041bf856cefa admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:29.606078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacaa5d2-180c-412c-80b0-041bf856cefa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:29.607998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacaa5d2-180c-412c-80b0-041bf856cefa admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:29.627791 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1170/2322] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:31:29 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:31:29.632192 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df83b8-ace9-43ad-9994-8b970dee4d1b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:29.643275 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df83b8-ace9-43ad-9994-8b970dee4d1b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:29.647905 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22250dd0-a78c-446c-ac96-9a50453a884e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:29.648559 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22250dd0-a78c-446c-ac96-9a50453a884e admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:29.648846 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22250dd0-a78c-446c-ac96-9a50453a884e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:29.649126 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22250dd0-a78c-446c-ac96-9a50453a884e admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:29.649890 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22250dd0-a78c-446c-ac96-9a50453a884e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:29.651523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22250dd0-a78c-446c-ac96-9a50453a884e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:29.662005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1153/2323] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:31:29.665901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacaa5d2-180c-412c-80b0-041bf856cefa admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:29.675860 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacaa5d2-180c-412c-80b0-041bf856cefa admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:29.680356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a9d09-95e5-45bb-a08a-2131c0c471ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:29.680999 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a9d09-95e5-45bb-a08a-2131c0c471ac admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:29.681281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a9d09-95e5-45bb-a08a-2131c0c471ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:29.681561 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a9d09-95e5-45bb-a08a-2131c0c471ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:29.682302 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a9d09-95e5-45bb-a08a-2131c0c471ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:29.683896 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a9d09-95e5-45bb-a08a-2131c0c471ac admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:29.694290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1171/2324] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:31:29.698319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22250dd0-a78c-446c-ac96-9a50453a884e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:29.705453 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:29.709665 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22250dd0-a78c-446c-ac96-9a50453a884e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:29.715329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bccaf6-af27-4d18-85c4-7050f5bbf892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:29.716237 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bccaf6-af27-4d18-85c4-7050f5bbf892 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:29.716670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bccaf6-af27-4d18-85c4-7050f5bbf892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:29.716951 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bccaf6-af27-4d18-85c4-7050f5bbf892 admin admin] PATH_INFO: `/v3/projects/cc479f51a55644e7b8bf505ff1798647/users/2b06e7db4946452b9e1f4baf99dfb942/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:29.731384 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bccaf6-af27-4d18-85c4-7050f5bbf892 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc479f51a55644e7b8bf505ff1798647, user_id=2b06e7db4946452b9e1f4baf99dfb942, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:29.734095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bccaf6-af27-4d18-85c4-7050f5bbf892 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:29.746467 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1154/2325] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:31:29 2022] PUT /identity/v3/projects/cc479f51a55644e7b8bf505ff1798647/users/2b06e7db4946452b9e1f4baf99dfb942/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:31:29.757448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1e9c1-fbf7-4922-85dc-d1a46b9c084e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:29.757779 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1e9c1-fbf7-4922-85dc-d1a46b9c084e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:29.758088 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1e9c1-fbf7-4922-85dc-d1a46b9c084e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:29.784393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-20b1e9c1-fbf7-4922-85dc-d1a46b9c084e None None] MFA Rules not processed for user `2b06e7db4946452b9e1f4baf99dfb942`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:29.785637 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-20b1e9c1-fbf7-4922-85dc-d1a46b9c084e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:29.798355 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:29.858248 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1172/2326] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:31:29 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:29.872511 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bccaf6-af27-4d18-85c4-7050f5bbf892 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:29.874356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a5bccaf6-af27-4d18-85c4-7050f5bbf892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:29.946694 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bccaf6-af27-4d18-85c4-7050f5bbf892 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:29.950784 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fcf2da-6c0b-4da9-ae5b-471440c8f7ae service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37a9fea702cd405ea9fc14ec2c689648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:29.951443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fcf2da-6c0b-4da9-ae5b-471440c8f7ae service glance] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:29.951730 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fcf2da-6c0b-4da9-ae5b-471440c8f7ae service glance] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:29.952009 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fcf2da-6c0b-4da9-ae5b-471440c8f7ae service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:29.961873 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fcf2da-6c0b-4da9-ae5b-471440c8f7ae service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:29.963340 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fcf2da-6c0b-4da9-ae5b-471440c8f7ae service glance] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:29.973874 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1155/2327] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:31:29 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:31:33.045098 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1e9c1-fbf7-4922-85dc-d1a46b9c084e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:33.046594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-20b1e9c1-fbf7-4922-85dc-d1a46b9c084e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:33.113258 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1e9c1-fbf7-4922-85dc-d1a46b9c084e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:33.117781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc1d7a-5313-40cd-953b-ae2153f2d4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:33.118659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc1d7a-5313-40cd-953b-ae2153f2d4d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:33.118952 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc1d7a-5313-40cd-953b-ae2153f2d4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:33.119235 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc1d7a-5313-40cd-953b-ae2153f2d4d2 admin admin] PATH_INFO: `/v3/users/d6f7d8cf91874e37a87852661d0a3f1a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:33.120441 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc1d7a-5313-40cd-953b-ae2153f2d4d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6f7d8cf91874e37a87852661d0a3f1a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:33.122425 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc1d7a-5313-40cd-953b-ae2153f2d4d2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:33.170176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-78cc1d7a-5313-40cd-953b-ae2153f2d4d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6f7d8cf91874e37a87852661d0a3f1a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:33.182101 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-78cc1d7a-5313-40cd-953b-ae2153f2d4d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6f7d8cf91874e37a87852661d0a3f1a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:33.182762 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-78cc1d7a-5313-40cd-953b-ae2153f2d4d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6f7d8cf91874e37a87852661d0a3f1a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:33.191197 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-78cc1d7a-5313-40cd-953b-ae2153f2d4d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6f7d8cf91874e37a87852661d0a3f1a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:33.192091 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-78cc1d7a-5313-40cd-953b-ae2153f2d4d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6f7d8cf91874e37a87852661d0a3f1a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:33.207534 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1173/2328] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:31:33 2022] DELETE /identity/v3/users/d6f7d8cf91874e37a87852661d0a3f1a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:31:33.211996 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfe68b-4809-4bea-800e-4e06de9245a6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:33.212453 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfe68b-4809-4bea-800e-4e06de9245a6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:33.212561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfe68b-4809-4bea-800e-4e06de9245a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:33.231438 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fccfe68b-4809-4bea-800e-4e06de9245a6 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:33.232855 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fccfe68b-4809-4bea-800e-4e06de9245a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:33.292945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1156/2329] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:33.303193 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc1d7a-5313-40cd-953b-ae2153f2d4d2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:33.304632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-78cc1d7a-5313-40cd-953b-ae2153f2d4d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:33.347075 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:33.371991 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc1d7a-5313-40cd-953b-ae2153f2d4d2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:33.375961 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49f612-c27b-4776-a099-425366cecb31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:33.379387 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49f612-c27b-4776-a099-425366cecb31 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:33.379387 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49f612-c27b-4776-a099-425366cecb31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:33.379387 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49f612-c27b-4776-a099-425366cecb31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:33.387142 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49f612-c27b-4776-a099-425366cecb31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:33.388464 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49f612-c27b-4776-a099-425366cecb31 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:33.398603 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1174/2330] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:31:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:31:33.704530 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfe68b-4809-4bea-800e-4e06de9245a6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:33.706261 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fccfe68b-4809-4bea-800e-4e06de9245a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:33.719121 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfe68b-4809-4bea-800e-4e06de9245a6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:33.724931 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa178f-363f-4473-9041-12d6d5f38f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:33.725880 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa178f-363f-4473-9041-12d6d5f38f12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:33.726167 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa178f-363f-4473-9041-12d6d5f38f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:33.726452 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa178f-363f-4473-9041-12d6d5f38f12 admin admin] PATH_INFO: `/v3/projects/5710d16b15c0442ea42d7313f87b887d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:33.727656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa178f-363f-4473-9041-12d6d5f38f12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5710d16b15c0442ea42d7313f87b887d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:33.733645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa178f-363f-4473-9041-12d6d5f38f12 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:33.783536 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e0aa178f-363f-4473-9041-12d6d5f38f12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5710d16b15c0442ea42d7313f87b887d', 'request_id': 'req-e0aa178f-363f-4473-9041-12d6d5f38f12'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:33.789605 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e0aa178f-363f-4473-9041-12d6d5f38f12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5710d16b15c0442ea42d7313f87b887d', 'request_id': 'req-e0aa178f-363f-4473-9041-12d6d5f38f12'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:33.795137 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e0aa178f-363f-4473-9041-12d6d5f38f12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5710d16b15c0442ea42d7313f87b887d', 'request_id': 'req-e0aa178f-363f-4473-9041-12d6d5f38f12'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:33.810086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e0aa178f-363f-4473-9041-12d6d5f38f12 admin admin] The token cache is being invalidate because project 5710d16b15c0442ea42d7313f87b887d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:31:33.810377 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e0aa178f-363f-4473-9041-12d6d5f38f12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:33.811055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e0aa178f-363f-4473-9041-12d6d5f38f12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:33.812612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1157/2331] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:31:33 2022] DELETE /identity/v3/projects/5710d16b15c0442ea42d7313f87b887d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:31:33.819711 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3cfd1-c3e9-415c-997b-b730d596191d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:33.820109 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3cfd1-c3e9-415c-997b-b730d596191d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:33.820751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3cfd1-c3e9-415c-997b-b730d596191d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:33.875884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-bed3cfd1-c3e9-415c-997b-b730d596191d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:33.878293 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bed3cfd1-c3e9-415c-997b-b730d596191d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:33.980891 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1175/2332] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:33.981211 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49c84e-d3d1-4b18-a841-91fda444c9f4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:33.981545 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49c84e-d3d1-4b18-a841-91fda444c9f4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:33.981857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49c84e-d3d1-4b18-a841-91fda444c9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:34.003411 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:34.011731 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6c49c84e-d3d1-4b18-a841-91fda444c9f4 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:34.014099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6c49c84e-d3d1-4b18-a841-91fda444c9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:34.030436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1158/2333] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:34.034585 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3cfd1-c3e9-415c-997b-b730d596191d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:34.050945 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3cfd1-c3e9-415c-997b-b730d596191d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:34.055637 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f94b6f-24d2-46ca-a05a-adaf0b71d4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:34.056439 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f94b6f-24d2-46ca-a05a-adaf0b71d4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:34.056730 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f94b6f-24d2-46ca-a05a-adaf0b71d4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:34.057017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f94b6f-24d2-46ca-a05a-adaf0b71d4e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:34.057764 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f94b6f-24d2-46ca-a05a-adaf0b71d4e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:31:34.058074 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f94b6f-24d2-46ca-a05a-adaf0b71d4e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:34.059802 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f94b6f-24d2-46ca-a05a-adaf0b71d4e0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:34.073530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1176/2334] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:31:34.077817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49c84e-d3d1-4b18-a841-91fda444c9f4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:34.087925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49c84e-d3d1-4b18-a841-91fda444c9f4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:34.092966 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f6576-f37a-43e6-b08d-1979aa928a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:34.093650 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f6576-f37a-43e6-b08d-1979aa928a2d admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:34.093982 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f6576-f37a-43e6-b08d-1979aa928a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:34.094370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f6576-f37a-43e6-b08d-1979aa928a2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:34.095148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f6576-f37a-43e6-b08d-1979aa928a2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:34.097300 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f6576-f37a-43e6-b08d-1979aa928a2d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:34.127632 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1159/2335] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:31:34 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:31:34.131814 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f94b6f-24d2-46ca-a05a-adaf0b71d4e0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:34.145227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f94b6f-24d2-46ca-a05a-adaf0b71d4e0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:34.238416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f72401-c8e4-491c-b200-937cf71e2a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:34.239130 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f72401-c8e4-491c-b200-937cf71e2a12 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:34.239420 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f72401-c8e4-491c-b200-937cf71e2a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:34.239704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f72401-c8e4-491c-b200-937cf71e2a12 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:34.240441 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f72401-c8e4-491c-b200-937cf71e2a12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:34.242474 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f72401-c8e4-491c-b200-937cf71e2a12 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:34.265992 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1177/2336] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:31:34 2022] POST /identity/v3/users => generated 287 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:31:34.270060 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f6576-f37a-43e6-b08d-1979aa928a2d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:34.282983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f6576-f37a-43e6-b08d-1979aa928a2d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:34.288536 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbc48e-eb4b-4646-b80a-7dbf6f082796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:34.289205 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbc48e-eb4b-4646-b80a-7dbf6f082796 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:34.289494 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbc48e-eb4b-4646-b80a-7dbf6f082796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:34.289811 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbc48e-eb4b-4646-b80a-7dbf6f082796 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:34.290546 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbc48e-eb4b-4646-b80a-7dbf6f082796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:34.292177 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbc48e-eb4b-4646-b80a-7dbf6f082796 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:34.305947 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1160/2337] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:31:34.309734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f72401-c8e4-491c-b200-937cf71e2a12 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:34.327513 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f72401-c8e4-491c-b200-937cf71e2a12 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:34.333488 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e389250-a569-486f-8889-6ff79bdeee99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:34.334264 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e389250-a569-486f-8889-6ff79bdeee99 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:34.334626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e389250-a569-486f-8889-6ff79bdeee99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:34.334987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e389250-a569-486f-8889-6ff79bdeee99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:34.335797 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e389250-a569-486f-8889-6ff79bdeee99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:34.337504 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e389250-a569-486f-8889-6ff79bdeee99 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:34.349262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1178/2338] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:31:34.353209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbc48e-eb4b-4646-b80a-7dbf6f082796 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:34.363445 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbc48e-eb4b-4646-b80a-7dbf6f082796 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:34.368555 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ac8be-c29d-4e4f-b54e-8173ca8afe8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:34.374287 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ac8be-c29d-4e4f-b54e-8173ca8afe8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:34.374287 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ac8be-c29d-4e4f-b54e-8173ca8afe8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:34.374287 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ac8be-c29d-4e4f-b54e-8173ca8afe8e admin admin] PATH_INFO: `/v3/projects/72283c6d9d6b4f529e8f195c8ebbca64/users/2d29f9c3387847b4bebaba3a1e3c0be8/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:34.387753 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ac8be-c29d-4e4f-b54e-8173ca8afe8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=72283c6d9d6b4f529e8f195c8ebbca64, user_id=2d29f9c3387847b4bebaba3a1e3c0be8, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:34.390774 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ac8be-c29d-4e4f-b54e-8173ca8afe8e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:34.405319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1161/2339] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:31:34 2022] PUT /identity/v3/projects/72283c6d9d6b4f529e8f195c8ebbca64/users/2d29f9c3387847b4bebaba3a1e3c0be8/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:31:34.416679 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573d283-46e0-46fd-9aee-aa5297514492 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:34.417061 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573d283-46e0-46fd-9aee-aa5297514492 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:34.417258 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573d283-46e0-46fd-9aee-aa5297514492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:34.449298 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6573d283-46e0-46fd-9aee-aa5297514492 None None] MFA Rules not processed for user `2d29f9c3387847b4bebaba3a1e3c0be8`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:34.450578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6573d283-46e0-46fd-9aee-aa5297514492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:34.490997 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:34.537691 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1179/2340] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:31:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:34.548847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ac8be-c29d-4e4f-b54e-8173ca8afe8e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:34.550519 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4e0ac8be-c29d-4e4f-b54e-8173ca8afe8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:34.668295 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ac8be-c29d-4e4f-b54e-8173ca8afe8e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:34.673086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6d7fd-cb60-4ca7-8abf-37ac064e318c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:34.673934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6d7fd-cb60-4ca7-8abf-37ac064e318c service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:34.674333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6d7fd-cb60-4ca7-8abf-37ac064e318c service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:34.674880 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6d7fd-cb60-4ca7-8abf-37ac064e318c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:34.686711 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6d7fd-cb60-4ca7-8abf-37ac064e318c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:34.689419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6d7fd-cb60-4ca7-8abf-37ac064e318c service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:34.701962 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1162/2341] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:31:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:31:37.211598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9b505-43a5-48cd-80c6-c2a0ea3ea041 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:37.212189 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9b505-43a5-48cd-80c6-c2a0ea3ea041 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:37.212189 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9b505-43a5-48cd-80c6-c2a0ea3ea041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:37.230696 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-47d9b505-43a5-48cd-80c6-c2a0ea3ea041 None None] MFA Rules not processed for user `2d29f9c3387847b4bebaba3a1e3c0be8`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:37.231909 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-47d9b505-43a5-48cd-80c6-c2a0ea3ea041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:37.238818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1180/2342] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:31:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:39.203936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6d7fd-cb60-4ca7-8abf-37ac064e318c service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:39.206483 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3cf6d7fd-cb60-4ca7-8abf-37ac064e318c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:39.329367 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9b505-43a5-48cd-80c6-c2a0ea3ea041 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:39.332774 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-47d9b505-43a5-48cd-80c6-c2a0ea3ea041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:39.369330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6d7fd-cb60-4ca7-8abf-37ac064e318c service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:39.375467 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b64ee6-b184-4dbd-8726-45c282018e35 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49325c1733174e0c98b3475a10c9b5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:39.377280 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b64ee6-b184-4dbd-8726-45c282018e35 service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:39.377568 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b64ee6-b184-4dbd-8726-45c282018e35 service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:39.377875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b64ee6-b184-4dbd-8726-45c282018e35 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:39.379693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b3b64ee6-b184-4dbd-8726-45c282018e35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:39.390726 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:39.395416 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b64ee6-b184-4dbd-8726-45c282018e35 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:39.398884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b64ee6-b184-4dbd-8726-45c282018e35 service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:39.413159 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1163/2343] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:31:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:31:39.424539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9b505-43a5-48cd-80c6-c2a0ea3ea041 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:39.429861 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e6019-bc93-41bb-b081-708a1827ccf6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49325c1733174e0c98b3475a10c9b5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:39.430759 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e6019-bc93-41bb-b081-708a1827ccf6 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:39.431151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e6019-bc93-41bb-b081-708a1827ccf6 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:39.431539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e6019-bc93-41bb-b081-708a1827ccf6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:39.441281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e6019-bc93-41bb-b081-708a1827ccf6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:39.442973 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e6019-bc93-41bb-b081-708a1827ccf6 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:39.453670 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1181/2344] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:31:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:31:49.407600 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:49.449418 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=47 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:32:09.234724 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2d597-0757-43b7-9d3e-63f878aea2d0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:09.235235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2d597-0757-43b7-9d3e-63f878aea2d0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:09.235340 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2d597-0757-43b7-9d3e-63f878aea2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:09.252127 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-cfd2d597-0757-43b7-9d3e-63f878aea2d0 None None] MFA Rules not processed for user `4128f8b7190e455b8dc599a404e3608c`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:32:09.254002 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cfd2d597-0757-43b7-9d3e-63f878aea2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:09.307474 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1164/2345] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:32:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:32:19.300460 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=25 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:32:26.121927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58d800-dc5a-414a-85c5-d8d12649d28d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:26.122407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58d800-dc5a-414a-85c5-d8d12649d28d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:26.122508 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58d800-dc5a-414a-85c5-d8d12649d28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:26.181469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7a58d800-dc5a-414a-85c5-d8d12649d28d None None] MFA Rules not processed for user `4ddcf6306e704fe9ad5ed8779178ae67`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:32:26.182708 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7a58d800-dc5a-414a-85c5-d8d12649d28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:26.254515 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1182/2346] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 30 18:32:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:32:26.298209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2d597-0757-43b7-9d3e-63f878aea2d0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:26.299816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cfd2d597-0757-43b7-9d3e-63f878aea2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:26.391417 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2d597-0757-43b7-9d3e-63f878aea2d0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:26.396984 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d985585b-2f6e-4a52-95f8-ee8a7870c8fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bd521b9e93f4b3b8af64d46f36c019e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:26.397686 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985585b-2f6e-4a52-95f8-ee8a7870c8fe service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:26.397970 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985585b-2f6e-4a52-95f8-ee8a7870c8fe service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:26.398245 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985585b-2f6e-4a52-95f8-ee8a7870c8fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:26.406626 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d985585b-2f6e-4a52-95f8-ee8a7870c8fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:26.408205 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d985585b-2f6e-4a52-95f8-ee8a7870c8fe service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:26.417294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1165/2347] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 30 18:32:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 18:32:36.250886 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=47 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:32:36.415546 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=41 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:32:39.203567 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97657ab0-8cba-457c-a3f6-3fea65c24673 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:39.204109 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97657ab0-8cba-457c-a3f6-3fea65c24673 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:39.204208 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97657ab0-8cba-457c-a3f6-3fea65c24673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:39.224095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-97657ab0-8cba-457c-a3f6-3fea65c24673 None None] MFA Rules not processed for user `4ddcf6306e704fe9ad5ed8779178ae67`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:32:39.225679 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-97657ab0-8cba-457c-a3f6-3fea65c24673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:39.233746 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1183/2348] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 30 18:32:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:32:39.249240 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d985585b-2f6e-4a52-95f8-ee8a7870c8fe service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:39.261402 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d985585b-2f6e-4a52-95f8-ee8a7870c8fe service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:39.267214 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3cb9f-a103-4017-8334-4a75a76e625e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bd521b9e93f4b3b8af64d46f36c019e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:39.268031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3cb9f-a103-4017-8334-4a75a76e625e service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:39.269291 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3cb9f-a103-4017-8334-4a75a76e625e service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:39.269884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3cb9f-a103-4017-8334-4a75a76e625e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:39.279480 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3cb9f-a103-4017-8334-4a75a76e625e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:39.281380 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3cb9f-a103-4017-8334-4a75a76e625e service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:39.292796 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1166/2349] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 30 18:32:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 18:32:40.036385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7bed1-07d1-4939-8465-1cfae9df31f0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:40.036693 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7bed1-07d1-4939-8465-1cfae9df31f0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:40.037353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7bed1-07d1-4939-8465-1cfae9df31f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:40.058144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7bd7bed1-07d1-4939-8465-1cfae9df31f0 None None] MFA Rules not processed for user `4ddcf6306e704fe9ad5ed8779178ae67`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:32:40.059958 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7bd7bed1-07d1-4939-8465-1cfae9df31f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:40.073158 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1184/2350] 10.222.0.33 () {58 vars in 967 bytes} [Thu Jun 30 18:32:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:32:40.093473 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3cb9f-a103-4017-8334-4a75a76e625e service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:40.108479 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3cb9f-a103-4017-8334-4a75a76e625e service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:40.114355 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95b2e4-d4d6-459f-abe6-d93c0786fbc5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bd521b9e93f4b3b8af64d46f36c019e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:40.115263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95b2e4-d4d6-459f-abe6-d93c0786fbc5 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:40.115749 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95b2e4-d4d6-459f-abe6-d93c0786fbc5 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:40.116333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95b2e4-d4d6-459f-abe6-d93c0786fbc5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:40.126455 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95b2e4-d4d6-459f-abe6-d93c0786fbc5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:40.129803 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95b2e4-d4d6-459f-abe6-d93c0786fbc5 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:40.141722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1167/2351] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 30 18:32:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 18:32:40.434557 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c233735-b55f-4a25-86ba-fb68f95643d6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:40.435044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c233735-b55f-4a25-86ba-fb68f95643d6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:40.435142 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c233735-b55f-4a25-86ba-fb68f95643d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:40.454319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-3c233735-b55f-4a25-86ba-fb68f95643d6 None None] MFA Rules not processed for user `4ddcf6306e704fe9ad5ed8779178ae67`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:32:40.455527 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3c233735-b55f-4a25-86ba-fb68f95643d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:40.463506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1185/2352] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 30 18:32:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:32:40.475333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95b2e4-d4d6-459f-abe6-d93c0786fbc5 service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:40.485998 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95b2e4-d4d6-459f-abe6-d93c0786fbc5 service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:40.492069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fa968-fcfa-4c89-a4bd-3005b1f05096 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bd521b9e93f4b3b8af64d46f36c019e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:40.492705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fa968-fcfa-4c89-a4bd-3005b1f05096 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:40.492994 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fa968-fcfa-4c89-a4bd-3005b1f05096 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:40.493361 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fa968-fcfa-4c89-a4bd-3005b1f05096 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:40.501550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435fa968-fcfa-4c89-a4bd-3005b1f05096 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:40.503912 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435fa968-fcfa-4c89-a4bd-3005b1f05096 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:40.514313 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1168/2353] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 30 18:32:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 18:32:40.970825 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167db2a-0856-4b9f-9c8c-95f9f3897238 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:40.970825 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167db2a-0856-4b9f-9c8c-95f9f3897238 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:40.970825 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167db2a-0856-4b9f-9c8c-95f9f3897238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:40.995740 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-5167db2a-0856-4b9f-9c8c-95f9f3897238 None None] MFA Rules not processed for user `4ddcf6306e704fe9ad5ed8779178ae67`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:32:40.996992 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5167db2a-0856-4b9f-9c8c-95f9f3897238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:41.005935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1186/2354] 10.222.0.33 () {58 vars in 967 bytes} [Thu Jun 30 18:32:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:32:41.020563 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fa968-fcfa-4c89-a4bd-3005b1f05096 service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:41.032554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fa968-fcfa-4c89-a4bd-3005b1f05096 service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:41.038372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40a566-d0cc-4ce0-abab-668b4d409385 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bd521b9e93f4b3b8af64d46f36c019e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:41.039037 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40a566-d0cc-4ce0-abab-668b4d409385 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:41.039317 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40a566-d0cc-4ce0-abab-668b4d409385 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:41.039589 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40a566-d0cc-4ce0-abab-668b4d409385 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:41.056716 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40a566-d0cc-4ce0-abab-668b4d409385 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:41.058418 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40a566-d0cc-4ce0-abab-668b4d409385 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:41.069415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1169/2355] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 30 18:32:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 18:32:43.369235 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167db2a-0856-4b9f-9c8c-95f9f3897238 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:43.440953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167db2a-0856-4b9f-9c8c-95f9f3897238 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:43.446784 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8b028-09d9-4deb-a96c-8b2b93853ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:43.448082 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8b028-09d9-4deb-a96c-8b2b93853ae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:43.448373 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8b028-09d9-4deb-a96c-8b2b93853ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:43.448675 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8b028-09d9-4deb-a96c-8b2b93853ae6 admin admin] PATH_INFO: `/v3/users/2d29f9c3387847b4bebaba3a1e3c0be8` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:43.449977 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8b028-09d9-4deb-a96c-8b2b93853ae6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d29f9c3387847b4bebaba3a1e3c0be8)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:43.451975 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8b028-09d9-4deb-a96c-8b2b93853ae6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:43.509669 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bec8b028-09d9-4deb-a96c-8b2b93853ae6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d29f9c3387847b4bebaba3a1e3c0be8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:43.534850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bec8b028-09d9-4deb-a96c-8b2b93853ae6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d29f9c3387847b4bebaba3a1e3c0be8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:43.536089 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bec8b028-09d9-4deb-a96c-8b2b93853ae6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d29f9c3387847b4bebaba3a1e3c0be8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:43.546125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bec8b028-09d9-4deb-a96c-8b2b93853ae6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d29f9c3387847b4bebaba3a1e3c0be8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:43.547086 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bec8b028-09d9-4deb-a96c-8b2b93853ae6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d29f9c3387847b4bebaba3a1e3c0be8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:43.562302 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1187/2356] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:32:43 2022] DELETE /identity/v3/users/2d29f9c3387847b4bebaba3a1e3c0be8 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:32:43.566828 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe95df-8eec-4730-b180-6a8956bbb1fb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:43.567732 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe95df-8eec-4730-b180-6a8956bbb1fb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:43.568013 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe95df-8eec-4730-b180-6a8956bbb1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:43.588120 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-49fe95df-8eec-4730-b180-6a8956bbb1fb None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:32:43.589379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-49fe95df-8eec-4730-b180-6a8956bbb1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:43.671131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1170/2357] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:32:43.686262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8b028-09d9-4deb-a96c-8b2b93853ae6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:43.687840 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bec8b028-09d9-4deb-a96c-8b2b93853ae6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:43.728924 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe95df-8eec-4730-b180-6a8956bbb1fb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:43.730671 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-49fe95df-8eec-4730-b180-6a8956bbb1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:43.745706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe95df-8eec-4730-b180-6a8956bbb1fb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:43.752504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889d9c3-960a-46c7-aba7-7961f587bd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:43.753450 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889d9c3-960a-46c7-aba7-7961f587bd49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:43.753789 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889d9c3-960a-46c7-aba7-7961f587bd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:43.754070 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889d9c3-960a-46c7-aba7-7961f587bd49 admin admin] PATH_INFO: `/v3/users/2b06e7db4946452b9e1f4baf99dfb942` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:43.756129 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889d9c3-960a-46c7-aba7-7961f587bd49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b06e7db4946452b9e1f4baf99dfb942)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:43.756267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8b028-09d9-4deb-a96c-8b2b93853ae6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:43.758280 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889d9c3-960a-46c7-aba7-7961f587bd49 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:43.759977 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7eb941-5d19-4322-a5f0-cd30dba905e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:43.760634 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7eb941-5d19-4322-a5f0-cd30dba905e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:43.760926 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7eb941-5d19-4322-a5f0-cd30dba905e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:43.761295 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7eb941-5d19-4322-a5f0-cd30dba905e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:43.775249 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:32:43.778992 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7eb941-5d19-4322-a5f0-cd30dba905e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:43.781139 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7eb941-5d19-4322-a5f0-cd30dba905e3 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:43.801644 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1188/2358] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:32:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:32:43.824209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3889d9c3-960a-46c7-aba7-7961f587bd49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b06e7db4946452b9e1f4baf99dfb942'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:43.824904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3889d9c3-960a-46c7-aba7-7961f587bd49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b06e7db4946452b9e1f4baf99dfb942'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:43.838753 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3889d9c3-960a-46c7-aba7-7961f587bd49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b06e7db4946452b9e1f4baf99dfb942'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:43.850493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3889d9c3-960a-46c7-aba7-7961f587bd49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b06e7db4946452b9e1f4baf99dfb942'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:43.851427 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3889d9c3-960a-46c7-aba7-7961f587bd49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b06e7db4946452b9e1f4baf99dfb942'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:43.870515 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1171/2359] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:32:43 2022] DELETE /identity/v3/users/2b06e7db4946452b9e1f4baf99dfb942 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:32:43.872177 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc3023-7c4a-4c65-a65e-974b9e2693b1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:43.872472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc3023-7c4a-4c65-a65e-974b9e2693b1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:43.872756 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc3023-7c4a-4c65-a65e-974b9e2693b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:43.893496 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c9cc3023-7c4a-4c65-a65e-974b9e2693b1 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:32:43.895698 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c9cc3023-7c4a-4c65-a65e-974b9e2693b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:43.982464 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1189/2360] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:32:43.993814 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889d9c3-960a-46c7-aba7-7961f587bd49 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:43.995755 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3889d9c3-960a-46c7-aba7-7961f587bd49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:44.010611 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc3023-7c4a-4c65-a65e-974b9e2693b1 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:44.014142 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c9cc3023-7c4a-4c65-a65e-974b9e2693b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:44.032263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:32:44.040958 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc3023-7c4a-4c65-a65e-974b9e2693b1 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:44.079397 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ad815-4087-4dfd-b4c9-ed607564ae26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:44.080431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ad815-4087-4dfd-b4c9-ed607564ae26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:44.080650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ad815-4087-4dfd-b4c9-ed607564ae26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:44.080902 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ad815-4087-4dfd-b4c9-ed607564ae26 admin admin] PATH_INFO: `/v3/projects/72283c6d9d6b4f529e8f195c8ebbca64` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:44.084098 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ad815-4087-4dfd-b4c9-ed607564ae26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72283c6d9d6b4f529e8f195c8ebbca64)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:44.086412 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ad815-4087-4dfd-b4c9-ed607564ae26 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:44.103127 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889d9c3-960a-46c7-aba7-7961f587bd49 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:44.107366 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af511476-f5a6-43e9-a19a-c859ae124575 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:44.108087 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af511476-f5a6-43e9-a19a-c859ae124575 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:44.108420 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af511476-f5a6-43e9-a19a-c859ae124575 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:44.108780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af511476-f5a6-43e9-a19a-c859ae124575 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:44.125753 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af511476-f5a6-43e9-a19a-c859ae124575 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:44.127261 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af511476-f5a6-43e9-a19a-c859ae124575 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:44.138776 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1172/2361] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:32:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:32:44.146319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ab0ad815-4087-4dfd-b4c9-ed607564ae26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72283c6d9d6b4f529e8f195c8ebbca64', 'request_id': 'req-ab0ad815-4087-4dfd-b4c9-ed607564ae26'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:44.148293 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b2321-fd87-416b-b9ec-3b88c360178d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:44.149618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b2321-fd87-416b-b9ec-3b88c360178d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:44.151260 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b2321-fd87-416b-b9ec-3b88c360178d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:44.154747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ab0ad815-4087-4dfd-b4c9-ed607564ae26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72283c6d9d6b4f529e8f195c8ebbca64', 'request_id': 'req-ab0ad815-4087-4dfd-b4c9-ed607564ae26'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:44.164292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ab0ad815-4087-4dfd-b4c9-ed607564ae26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72283c6d9d6b4f529e8f195c8ebbca64', 'request_id': 'req-ab0ad815-4087-4dfd-b4c9-ed607564ae26'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:44.177322 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5a0b2321-fd87-416b-b9ec-3b88c360178d None None] MFA Rules not processed for user `4ddcf6306e704fe9ad5ed8779178ae67`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:32:44.178807 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5a0b2321-fd87-416b-b9ec-3b88c360178d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:44.191003 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ab0ad815-4087-4dfd-b4c9-ed607564ae26 admin admin] The token cache is being invalidate because project 72283c6d9d6b4f529e8f195c8ebbca64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:32:44.192010 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ab0ad815-4087-4dfd-b4c9-ed607564ae26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:44.193686 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ab0ad815-4087-4dfd-b4c9-ed607564ae26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:44.221608 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1190/2362] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:32:44 2022] DELETE /identity/v3/projects/72283c6d9d6b4f529e8f195c8ebbca64 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:32:44.282021 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1173/2363] 10.222.0.33 () {58 vars in 967 bytes} [Thu Jun 30 18:32:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:32:44.300489 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ad815-4087-4dfd-b4c9-ed607564ae26 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:44.302118 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ab0ad815-4087-4dfd-b4c9-ed607564ae26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:44.435922 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ad815-4087-4dfd-b4c9-ed607564ae26 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:44.442957 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d81a8-34e1-4630-88e7-4791f086ecc8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bd521b9e93f4b3b8af64d46f36c019e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:44.443621 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d81a8-34e1-4630-88e7-4791f086ecc8 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:44.443911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d81a8-34e1-4630-88e7-4791f086ecc8 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:44.444190 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d81a8-34e1-4630-88e7-4791f086ecc8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:44.453111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d81a8-34e1-4630-88e7-4791f086ecc8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:44.454739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d81a8-34e1-4630-88e7-4791f086ecc8 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:44.463796 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1191/2364] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 30 18:32:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 18:32:44.512172 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b2321-fd87-416b-b9ec-3b88c360178d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:44.514163 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5a0b2321-fd87-416b-b9ec-3b88c360178d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:44.593360 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b2321-fd87-416b-b9ec-3b88c360178d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:44.599195 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd40e6-4b4a-4b78-98c1-899845a9a6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:44.600067 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd40e6-4b4a-4b78-98c1-899845a9a6f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:44.600355 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd40e6-4b4a-4b78-98c1-899845a9a6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:44.600636 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd40e6-4b4a-4b78-98c1-899845a9a6f9 admin admin] PATH_INFO: `/v3/projects/cc479f51a55644e7b8bf505ff1798647` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:44.601918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd40e6-4b4a-4b78-98c1-899845a9a6f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc479f51a55644e7b8bf505ff1798647)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:44.603848 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd40e6-4b4a-4b78-98c1-899845a9a6f9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:44.657188 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e1bd40e6-4b4a-4b78-98c1-899845a9a6f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc479f51a55644e7b8bf505ff1798647', 'request_id': 'req-e1bd40e6-4b4a-4b78-98c1-899845a9a6f9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:44.664267 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e1bd40e6-4b4a-4b78-98c1-899845a9a6f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc479f51a55644e7b8bf505ff1798647', 'request_id': 'req-e1bd40e6-4b4a-4b78-98c1-899845a9a6f9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:44.670298 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e1bd40e6-4b4a-4b78-98c1-899845a9a6f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc479f51a55644e7b8bf505ff1798647', 'request_id': 'req-e1bd40e6-4b4a-4b78-98c1-899845a9a6f9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:44.680280 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:32:44.686171 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e1bd40e6-4b4a-4b78-98c1-899845a9a6f9 admin admin] The token cache is being invalidate because project cc479f51a55644e7b8bf505ff1798647 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:32:44.686794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e1bd40e6-4b4a-4b78-98c1-899845a9a6f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:44.687697 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e1bd40e6-4b4a-4b78-98c1-899845a9a6f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:44.701614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1174/2365] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:32:44 2022] DELETE /identity/v3/projects/cc479f51a55644e7b8bf505ff1798647 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:32:46.498253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d81a8-34e1-4630-88e7-4791f086ecc8 service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:46.499777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a12d81a8-34e1-4630-88e7-4791f086ecc8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:46.518725 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d81a8-34e1-4630-88e7-4791f086ecc8 service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:46.524436 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:32:46.577634 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a11fbf-c1f6-452a-ac43-4da36ea80929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:46.578426 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a11fbf-c1f6-452a-ac43-4da36ea80929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:46.578713 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a11fbf-c1f6-452a-ac43-4da36ea80929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:46.578990 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a11fbf-c1f6-452a-ac43-4da36ea80929 admin admin] PATH_INFO: `/v3/users/374555cda1b4424894093ebcc51cd651` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:46.580177 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a11fbf-c1f6-452a-ac43-4da36ea80929 admin admin] RBAC: Authorizing `identity:delete_user(user_id=374555cda1b4424894093ebcc51cd651)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:46.582155 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a11fbf-c1f6-452a-ac43-4da36ea80929 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:46.628728 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-83a11fbf-c1f6-452a-ac43-4da36ea80929 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '374555cda1b4424894093ebcc51cd651'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:46.641381 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-83a11fbf-c1f6-452a-ac43-4da36ea80929 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '374555cda1b4424894093ebcc51cd651'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:46.642077 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-83a11fbf-c1f6-452a-ac43-4da36ea80929 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '374555cda1b4424894093ebcc51cd651'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:46.649850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-83a11fbf-c1f6-452a-ac43-4da36ea80929 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '374555cda1b4424894093ebcc51cd651'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:46.650477 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-83a11fbf-c1f6-452a-ac43-4da36ea80929 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '374555cda1b4424894093ebcc51cd651'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:46.663409 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1192/2366] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:32:46 2022] DELETE /identity/v3/users/374555cda1b4424894093ebcc51cd651 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:32:46.848646 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd40e6-4b4a-4b78-98c1-899845a9a6f9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:46.851170 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e1bd40e6-4b4a-4b78-98c1-899845a9a6f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:46.906383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd40e6-4b4a-4b78-98c1-899845a9a6f9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:46.966298 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de3790-d52f-4f34-a018-748089e2c562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:46.967158 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de3790-d52f-4f34-a018-748089e2c562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:46.967444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de3790-d52f-4f34-a018-748089e2c562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:46.967723 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de3790-d52f-4f34-a018-748089e2c562 admin admin] PATH_INFO: `/v3/projects/116a07f168ef45e1b2f4e7a084fba32c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:46.968915 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de3790-d52f-4f34-a018-748089e2c562 admin admin] RBAC: Authorizing `identity:delete_project(project_id=116a07f168ef45e1b2f4e7a084fba32c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:46.970864 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de3790-d52f-4f34-a018-748089e2c562 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:47.017896 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-33de3790-d52f-4f34-a018-748089e2c562 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '116a07f168ef45e1b2f4e7a084fba32c', 'request_id': 'req-33de3790-d52f-4f34-a018-748089e2c562'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:47.024752 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-33de3790-d52f-4f34-a018-748089e2c562 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '116a07f168ef45e1b2f4e7a084fba32c', 'request_id': 'req-33de3790-d52f-4f34-a018-748089e2c562'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:47.030337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-33de3790-d52f-4f34-a018-748089e2c562 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '116a07f168ef45e1b2f4e7a084fba32c', 'request_id': 'req-33de3790-d52f-4f34-a018-748089e2c562'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:47.042682 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-33de3790-d52f-4f34-a018-748089e2c562 admin admin] The token cache is being invalidate because project 116a07f168ef45e1b2f4e7a084fba32c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:32:47.042970 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-33de3790-d52f-4f34-a018-748089e2c562 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:47.043588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-33de3790-d52f-4f34-a018-748089e2c562 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:47.045154 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1175/2367] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:32:46 2022] DELETE /identity/v3/projects/116a07f168ef45e1b2f4e7a084fba32c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:32:56.659956 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:32:57.040164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=52,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:33:20.979694 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28945f-d180-4f1a-a923-338f6b3331a8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:20.980222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28945f-d180-4f1a-a923-338f6b3331a8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:20.980331 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28945f-d180-4f1a-a923-338f6b3331a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:20.999493 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cd28945f-d180-4f1a-a923-338f6b3331a8 None None] MFA Rules not processed for user `4ddcf6306e704fe9ad5ed8779178ae67`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:33:21.000712 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cd28945f-d180-4f1a-a923-338f6b3331a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:21.060730 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1193/2368] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 30 18:33:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:33:21.074259 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de3790-d52f-4f34-a018-748089e2c562 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:21.076611 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-33de3790-d52f-4f34-a018-748089e2c562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:21.155964 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de3790-d52f-4f34-a018-748089e2c562 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:21.161675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dded31a-ccc8-4e57-a6ee-8187636386bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bd521b9e93f4b3b8af64d46f36c019e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:21.162325 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dded31a-ccc8-4e57-a6ee-8187636386bc service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:21.162609 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dded31a-ccc8-4e57-a6ee-8187636386bc service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:21.162884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dded31a-ccc8-4e57-a6ee-8187636386bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:21.173847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dded31a-ccc8-4e57-a6ee-8187636386bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:21.175338 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dded31a-ccc8-4e57-a6ee-8187636386bc service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:21.190465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1176/2369] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 30 18:33:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 18:33:25.384661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28945f-d180-4f1a-a923-338f6b3331a8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:25.386869 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cd28945f-d180-4f1a-a923-338f6b3331a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:25.456960 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28945f-d180-4f1a-a923-338f6b3331a8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:25.460965 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef2a95-81d6-4fb2-a315-830affed63af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:25.461613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef2a95-81d6-4fb2-a315-830affed63af service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:25.461918 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef2a95-81d6-4fb2-a315-830affed63af service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:25.462202 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef2a95-81d6-4fb2-a315-830affed63af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:25.464009 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-47ef2a95-81d6-4fb2-a315-830affed63af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:25.586499 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef2a95-81d6-4fb2-a315-830affed63af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:25.589200 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef2a95-81d6-4fb2-a315-830affed63af service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:25.620042 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1194/2370] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:33:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:33:29.218606 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dded31a-ccc8-4e57-a6ee-8187636386bc service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:29.229180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dded31a-ccc8-4e57-a6ee-8187636386bc service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:29.236143 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013d96f4-2ff8-4520-a5fa-e1d03c5a9710 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:29.236792 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d96f4-2ff8-4520-a5fa-e1d03c5a9710 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:29.237075 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d96f4-2ff8-4520-a5fa-e1d03c5a9710 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:29.237484 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d96f4-2ff8-4520-a5fa-e1d03c5a9710 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:29.239634 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-013d96f4-2ff8-4520-a5fa-e1d03c5a9710 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:29.254521 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013d96f4-2ff8-4520-a5fa-e1d03c5a9710 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:29.256988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013d96f4-2ff8-4520-a5fa-e1d03c5a9710 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:29.273775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1177/2371] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:33:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:33:35.232398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef2a95-81d6-4fb2-a315-830affed63af service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:35.234822 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-47ef2a95-81d6-4fb2-a315-830affed63af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:35.248267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef2a95-81d6-4fb2-a315-830affed63af service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:35.254891 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf3468-70e2-4375-9156-c7c47e3cd950 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:35.255527 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf3468-70e2-4375-9156-c7c47e3cd950 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:35.255817 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf3468-70e2-4375-9156-c7c47e3cd950 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:35.256100 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf3468-70e2-4375-9156-c7c47e3cd950 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:35.257819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-dbcf3468-70e2-4375-9156-c7c47e3cd950 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:35.284632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:33:35.320840 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf3468-70e2-4375-9156-c7c47e3cd950 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:35.322393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf3468-70e2-4375-9156-c7c47e3cd950 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:35.331655 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1195/2372] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:33:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:33:35.986321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013d96f4-2ff8-4520-a5fa-e1d03c5a9710 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:35.995592 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013d96f4-2ff8-4520-a5fa-e1d03c5a9710 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:35.999928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc16e8-dbb9-4341-ad3d-fd3686c492ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:36.000575 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc16e8-dbb9-4341-ad3d-fd3686c492ba service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:36.000879 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc16e8-dbb9-4341-ad3d-fd3686c492ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:36.001158 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc16e8-dbb9-4341-ad3d-fd3686c492ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:36.003338 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d6bc16e8-dbb9-4341-ad3d-fd3686c492ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:36.012956 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc16e8-dbb9-4341-ad3d-fd3686c492ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:36.014964 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc16e8-dbb9-4341-ad3d-fd3686c492ba service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:36.025499 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1178/2373] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:33:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:33:36.128295 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee24e2-5d84-4f9e-8d37-c25cdafd11cc None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:36.128586 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee24e2-5d84-4f9e-8d37-c25cdafd11cc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:36.128926 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee24e2-5d84-4f9e-8d37-c25cdafd11cc None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:36.130408 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1196/2374] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 30 18:33:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:33:36.136558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6801cb5-47b0-4b27-b5d7-2502a74ce013 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:36.136844 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6801cb5-47b0-4b27-b5d7-2502a74ce013 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:36.137120 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6801cb5-47b0-4b27-b5d7-2502a74ce013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:36.175951 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c6801cb5-47b0-4b27-b5d7-2502a74ce013 None None] MFA Rules not processed for user `b8751a4e92174d18b7b138392d2579a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:33:36.177294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c6801cb5-47b0-4b27-b5d7-2502a74ce013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:36.187468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1179/2375] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 30 18:33:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:33:36.198842 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee24e2-5d84-4f9e-8d37-c25cdafd11cc None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:36.200944 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-61ee24e2-5d84-4f9e-8d37-c25cdafd11cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:36.291085 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee24e2-5d84-4f9e-8d37-c25cdafd11cc None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:36.295664 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e5268-88bc-4380-b037-0a5c7fc53581 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37a9fea702cd405ea9fc14ec2c689648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:36.296500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e5268-88bc-4380-b037-0a5c7fc53581 service glance] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:36.296821 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e5268-88bc-4380-b037-0a5c7fc53581 service glance] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:36.297206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e5268-88bc-4380-b037-0a5c7fc53581 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:36.308653 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e5268-88bc-4380-b037-0a5c7fc53581 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:36.310174 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e5268-88bc-4380-b037-0a5c7fc53581 service glance] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:36.321845 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1197/2376] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:33:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:33:36.389033 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aafed4-d2e8-40e6-ab68-8af0657dc306 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:36.390055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aafed4-d2e8-40e6-ab68-8af0657dc306 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:36.390336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aafed4-d2e8-40e6-ab68-8af0657dc306 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:36.391728 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1180/2377] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 30 18:33:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:33:36.396533 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb297967-08ed-49e2-bebf-2464586bcc4c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:36.396881 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb297967-08ed-49e2-bebf-2464586bcc4c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:36.397177 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb297967-08ed-49e2-bebf-2464586bcc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:36.415349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-cb297967-08ed-49e2-bebf-2464586bcc4c None None] MFA Rules not processed for user `1d8c54102fb340859ed0acc3189fe376`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:33:36.416556 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cb297967-08ed-49e2-bebf-2464586bcc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:36.423115 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1198/2378] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 30 18:33:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:33:36.434512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aafed4-d2e8-40e6-ab68-8af0657dc306 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:36.436737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-99aafed4-d2e8-40e6-ab68-8af0657dc306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:36.449857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aafed4-d2e8-40e6-ab68-8af0657dc306 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:36.454368 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635d818-78b8-4f04-aae4-13270779971d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:36.454986 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635d818-78b8-4f04-aae4-13270779971d service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:36.455271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635d818-78b8-4f04-aae4-13270779971d service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:36.455551 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635d818-78b8-4f04-aae4-13270779971d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:36.462642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f635d818-78b8-4f04-aae4-13270779971d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:36.464070 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f635d818-78b8-4f04-aae4-13270779971d service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:36.471868 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1181/2379] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:33:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:33:36.559876 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38de27-7af4-4958-8e61-25f6878133fc None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:36.560169 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38de27-7af4-4958-8e61-25f6878133fc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:36.560463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38de27-7af4-4958-8e61-25f6878133fc None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:36.562729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1199/2380] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 30 18:33:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:33:36.568130 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994771c-239c-4873-9623-65dc7d59a456 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:36.568423 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994771c-239c-4873-9623-65dc7d59a456 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:36.568705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994771c-239c-4873-9623-65dc7d59a456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:36.588717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2994771c-239c-4873-9623-65dc7d59a456 None None] MFA Rules not processed for user `b8751a4e92174d18b7b138392d2579a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:33:36.589963 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2994771c-239c-4873-9623-65dc7d59a456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:36.601205 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1182/2381] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 30 18:33:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:33:36.611811 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38de27-7af4-4958-8e61-25f6878133fc None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:36.622565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38de27-7af4-4958-8e61-25f6878133fc None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:36.626627 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c429701-6c51-4cc0-a287-3e01f5d9234c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:36.627249 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c429701-6c51-4cc0-a287-3e01f5d9234c service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:36.627539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c429701-6c51-4cc0-a287-3e01f5d9234c service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:36.627818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c429701-6c51-4cc0-a287-3e01f5d9234c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:36.638072 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c429701-6c51-4cc0-a287-3e01f5d9234c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:36.640202 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c429701-6c51-4cc0-a287-3e01f5d9234c service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:36.652134 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1200/2382] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:33:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:33:36.714879 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994771c-239c-4873-9623-65dc7d59a456 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:36.716267 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2994771c-239c-4873-9623-65dc7d59a456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:36.782642 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994771c-239c-4873-9623-65dc7d59a456 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:36.787745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aaba97-5a57-47b0-ad70-3ed1ab697343 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49325c1733174e0c98b3475a10c9b5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:36.788383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aaba97-5a57-47b0-ad70-3ed1ab697343 service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:36.788700 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aaba97-5a57-47b0-ad70-3ed1ab697343 service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:36.788931 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aaba97-5a57-47b0-ad70-3ed1ab697343 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:36.790827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-43aaba97-5a57-47b0-ad70-3ed1ab697343 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:36.802915 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aaba97-5a57-47b0-ad70-3ed1ab697343 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:36.804548 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aaba97-5a57-47b0-ad70-3ed1ab697343 service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:36.814899 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1183/2383] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:33:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:33:39.446486 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00432b-2c7e-4897-92ed-a0d97923eaa9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:39.447071 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00432b-2c7e-4897-92ed-a0d97923eaa9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:39.447071 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00432b-2c7e-4897-92ed-a0d97923eaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:39.465697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6f00432b-2c7e-4897-92ed-a0d97923eaa9 None None] MFA Rules not processed for user `4ddcf6306e704fe9ad5ed8779178ae67`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:33:39.466888 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6f00432b-2c7e-4897-92ed-a0d97923eaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:39.474544 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1201/2384] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 30 18:33:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:33:39.486882 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aaba97-5a57-47b0-ad70-3ed1ab697343 service placement] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:39.497911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aaba97-5a57-47b0-ad70-3ed1ab697343 service placement] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:39.502946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91259132-3040-4802-bbcb-2a2e98989b5f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bd521b9e93f4b3b8af64d46f36c019e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:39.503567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91259132-3040-4802-bbcb-2a2e98989b5f service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:39.503847 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91259132-3040-4802-bbcb-2a2e98989b5f service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:39.504124 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91259132-3040-4802-bbcb-2a2e98989b5f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:39.511921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91259132-3040-4802-bbcb-2a2e98989b5f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:39.513485 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91259132-3040-4802-bbcb-2a2e98989b5f service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:39.518204 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:33:39.522592 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1184/2385] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 30 18:33:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 18:33:40.417076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ee804-7a26-4c8f-be22-f082716a484b None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:40.417505 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ee804-7a26-4c8f-be22-f082716a484b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:40.417922 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ee804-7a26-4c8f-be22-f082716a484b None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:40.419459 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1202/2386] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 30 18:33:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:33:40.431618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec31508-2137-4d48-9a00-6f66c2a8e9ea None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:40.431915 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec31508-2137-4d48-9a00-6f66c2a8e9ea None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:40.432203 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec31508-2137-4d48-9a00-6f66c2a8e9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:40.492179 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fec31508-2137-4d48-9a00-6f66c2a8e9ea None None] MFA Rules not processed for user `49325c1733174e0c98b3475a10c9b5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:33:40.494357 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fec31508-2137-4d48-9a00-6f66c2a8e9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:40.506896 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1185/2387] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 30 18:33:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:33:40.509464 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ee804-7a26-4c8f-be22-f082716a484b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:40.511235 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5d8ee804-7a26-4c8f-be22-f082716a484b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:40.530248 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec31508-2137-4d48-9a00-6f66c2a8e9ea None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:40.544094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec31508-2137-4d48-9a00-6f66c2a8e9ea None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:40.562235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0251b-72e3-467b-a4e2-2a21c2ab02d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49325c1733174e0c98b3475a10c9b5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:40.566056 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0251b-72e3-467b-a4e2-2a21c2ab02d0 service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:40.566487 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0251b-72e3-467b-a4e2-2a21c2ab02d0 service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:40.566807 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0251b-72e3-467b-a4e2-2a21c2ab02d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:40.585392 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e0251b-72e3-467b-a4e2-2a21c2ab02d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:40.587297 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e0251b-72e3-467b-a4e2-2a21c2ab02d0 service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:40.598225 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1186/2388] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:33:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:33:40.599458 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ee804-7a26-4c8f-be22-f082716a484b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:40.604727 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e8e41-b477-492d-a7e5-31358a104397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:40.605656 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e8e41-b477-492d-a7e5-31358a104397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:40.605944 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e8e41-b477-492d-a7e5-31358a104397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:40.606220 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e8e41-b477-492d-a7e5-31358a104397 admin admin] PATH_INFO: `/v3/users/4128f8b7190e455b8dc599a404e3608c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:40.607433 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e8e41-b477-492d-a7e5-31358a104397 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4128f8b7190e455b8dc599a404e3608c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:40.609357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e8e41-b477-492d-a7e5-31358a104397 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:40.640919 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:33:40.668203 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-315e8e41-b477-492d-a7e5-31358a104397 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4128f8b7190e455b8dc599a404e3608c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:33:40.681020 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-315e8e41-b477-492d-a7e5-31358a104397 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4128f8b7190e455b8dc599a404e3608c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:33:40.681788 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-315e8e41-b477-492d-a7e5-31358a104397 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4128f8b7190e455b8dc599a404e3608c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:33:40.689133 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-315e8e41-b477-492d-a7e5-31358a104397 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4128f8b7190e455b8dc599a404e3608c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:33:40.690558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-315e8e41-b477-492d-a7e5-31358a104397 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4128f8b7190e455b8dc599a404e3608c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:33:40.703682 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1203/2389] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:33:40 2022] DELETE /identity/v3/users/4128f8b7190e455b8dc599a404e3608c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:33:40.891780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0251b-72e3-467b-a4e2-2a21c2ab02d0 service placement] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:40.902858 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0251b-72e3-467b-a4e2-2a21c2ab02d0 service placement] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:40.990942 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09c3eb-e65c-4890-9c9c-0e43214854ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:40.991951 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09c3eb-e65c-4890-9c9c-0e43214854ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:40.992297 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09c3eb-e65c-4890-9c9c-0e43214854ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:40.992640 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09c3eb-e65c-4890-9c9c-0e43214854ec admin admin] PATH_INFO: `/v3/projects/175743f30d314455a3bead1c50750fbe` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:40.993958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09c3eb-e65c-4890-9c9c-0e43214854ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=175743f30d314455a3bead1c50750fbe)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:40.995937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09c3eb-e65c-4890-9c9c-0e43214854ec admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:41.052517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3e09c3eb-e65c-4890-9c9c-0e43214854ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '175743f30d314455a3bead1c50750fbe', 'request_id': 'req-3e09c3eb-e65c-4890-9c9c-0e43214854ec'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:33:41.059560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3e09c3eb-e65c-4890-9c9c-0e43214854ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '175743f30d314455a3bead1c50750fbe', 'request_id': 'req-3e09c3eb-e65c-4890-9c9c-0e43214854ec'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:33:41.065733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3e09c3eb-e65c-4890-9c9c-0e43214854ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '175743f30d314455a3bead1c50750fbe', 'request_id': 'req-3e09c3eb-e65c-4890-9c9c-0e43214854ec'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:33:41.079343 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3e09c3eb-e65c-4890-9c9c-0e43214854ec admin admin] The token cache is being invalidate because project 175743f30d314455a3bead1c50750fbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:33:41.079739 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3e09c3eb-e65c-4890-9c9c-0e43214854ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:33:41.080467 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3e09c3eb-e65c-4890-9c9c-0e43214854ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:33:41.082359 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1187/2390] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:33:40 2022] DELETE /identity/v3/projects/175743f30d314455a3bead1c50750fbe => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:33:41.088464 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d659e-eeca-46c1-82d7-cadde0b9ce99 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:41.088464 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d659e-eeca-46c1-82d7-cadde0b9ce99 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:41.088464 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d659e-eeca-46c1-82d7-cadde0b9ce99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:41.109687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-821d659e-eeca-46c1-82d7-cadde0b9ce99 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:33:41.111751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-821d659e-eeca-46c1-82d7-cadde0b9ce99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:41.184751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1204/2391] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:33:41.193285 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48aeeef-f9e2-4f00-b6d9-8d0ff3368bf4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:41.193592 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48aeeef-f9e2-4f00-b6d9-8d0ff3368bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:41.193889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48aeeef-f9e2-4f00-b6d9-8d0ff3368bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:41.216515 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-c48aeeef-f9e2-4f00-b6d9-8d0ff3368bf4 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:33:41.217934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c48aeeef-f9e2-4f00-b6d9-8d0ff3368bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:41.233480 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1188/2392] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:33:41.238253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d659e-eeca-46c1-82d7-cadde0b9ce99 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:41.248321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d659e-eeca-46c1-82d7-cadde0b9ce99 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:41.253841 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07bde7e-9f03-45ca-b443-026c8efdb278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:41.254511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bde7e-9f03-45ca-b443-026c8efdb278 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:41.254803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bde7e-9f03-45ca-b443-026c8efdb278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:41.255097 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bde7e-9f03-45ca-b443-026c8efdb278 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:41.255827 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07bde7e-9f03-45ca-b443-026c8efdb278 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:33:41.256138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07bde7e-9f03-45ca-b443-026c8efdb278 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:41.257808 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07bde7e-9f03-45ca-b443-026c8efdb278 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:41.271154 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1205/2393] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:33:41.274836 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48aeeef-f9e2-4f00-b6d9-8d0ff3368bf4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:41.284960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48aeeef-f9e2-4f00-b6d9-8d0ff3368bf4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:41.290024 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616435ad-659f-4f26-9571-9f878e0ad56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:41.290681 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616435ad-659f-4f26-9571-9f878e0ad56f admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:41.290972 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616435ad-659f-4f26-9571-9f878e0ad56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:41.291248 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616435ad-659f-4f26-9571-9f878e0ad56f admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:41.291962 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616435ad-659f-4f26-9571-9f878e0ad56f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:41.293941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616435ad-659f-4f26-9571-9f878e0ad56f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:41.322890 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1189/2394] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:33:41 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:33:41.327413 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07bde7e-9f03-45ca-b443-026c8efdb278 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:41.337816 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07bde7e-9f03-45ca-b443-026c8efdb278 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:41.408683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915b07d-526f-46a7-916c-ebb9853c44f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:41.409821 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915b07d-526f-46a7-916c-ebb9853c44f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:41.410182 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915b07d-526f-46a7-916c-ebb9853c44f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:41.410534 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915b07d-526f-46a7-916c-ebb9853c44f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:41.411313 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d915b07d-526f-46a7-916c-ebb9853c44f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:41.413303 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d915b07d-526f-46a7-916c-ebb9853c44f4 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:41.440664 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1206/2395] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:33:41 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:33:41.441867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616435ad-659f-4f26-9571-9f878e0ad56f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:41.453071 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616435ad-659f-4f26-9571-9f878e0ad56f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:41.459027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508df96e-1177-46c5-b2c2-8533589e19a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:41.459699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508df96e-1177-46c5-b2c2-8533589e19a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:41.459986 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508df96e-1177-46c5-b2c2-8533589e19a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:41.460267 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508df96e-1177-46c5-b2c2-8533589e19a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:41.461000 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508df96e-1177-46c5-b2c2-8533589e19a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:41.462658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508df96e-1177-46c5-b2c2-8533589e19a6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:41.474683 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1190/2396] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:33:41.478709 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915b07d-526f-46a7-916c-ebb9853c44f4 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:41.488840 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915b07d-526f-46a7-916c-ebb9853c44f4 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:41.493446 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a6f07-b337-4556-b77c-c7cd47002064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:41.494135 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a6f07-b337-4556-b77c-c7cd47002064 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:41.494419 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a6f07-b337-4556-b77c-c7cd47002064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:41.494697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a6f07-b337-4556-b77c-c7cd47002064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:41.495416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a6f07-b337-4556-b77c-c7cd47002064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:41.497020 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a6f07-b337-4556-b77c-c7cd47002064 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:41.508858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1207/2397] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:33:41.513287 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508df96e-1177-46c5-b2c2-8533589e19a6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:41.524926 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508df96e-1177-46c5-b2c2-8533589e19a6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:41.529795 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380cce2-dc2b-4157-bc68-36f84dfcc4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:41.530682 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380cce2-dc2b-4157-bc68-36f84dfcc4d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:41.530963 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380cce2-dc2b-4157-bc68-36f84dfcc4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:41.531337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380cce2-dc2b-4157-bc68-36f84dfcc4d1 admin admin] PATH_INFO: `/v3/projects/64655c9e82bb4515a976f41ac93b242f/users/31c0594d0be84154a601d625bca699cd/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:41.549937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f380cce2-dc2b-4157-bc68-36f84dfcc4d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64655c9e82bb4515a976f41ac93b242f, user_id=31c0594d0be84154a601d625bca699cd, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:41.552730 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f380cce2-dc2b-4157-bc68-36f84dfcc4d1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:41.555354 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:33:41.571101 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1191/2398] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:33:41 2022] PUT /identity/v3/projects/64655c9e82bb4515a976f41ac93b242f/users/31c0594d0be84154a601d625bca699cd/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:33:41.576449 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fb5e5-fe55-441f-8b72-a55dc6e3cdbc None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:41.576909 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fb5e5-fe55-441f-8b72-a55dc6e3cdbc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:41.577025 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fb5e5-fe55-441f-8b72-a55dc6e3cdbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:41.590353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:33:41.597152 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ca3fb5e5-fe55-441f-8b72-a55dc6e3cdbc None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:33:41.599078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ca3fb5e5-fe55-441f-8b72-a55dc6e3cdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:41.660540 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1208/2399] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:33:41.670218 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380cce2-dc2b-4157-bc68-36f84dfcc4d1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:41.671988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f380cce2-dc2b-4157-bc68-36f84dfcc4d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:41.731374 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380cce2-dc2b-4157-bc68-36f84dfcc4d1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:41.736975 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90c81f-d5ff-4142-a0b5-09ab9abe81ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:41.737819 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90c81f-d5ff-4142-a0b5-09ab9abe81ae service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:41.738096 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90c81f-d5ff-4142-a0b5-09ab9abe81ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:41.738367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90c81f-d5ff-4142-a0b5-09ab9abe81ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:41.750344 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90c81f-d5ff-4142-a0b5-09ab9abe81ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:41.752324 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90c81f-d5ff-4142-a0b5-09ab9abe81ae service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:41.766025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1192/2400] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:33:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:33:46.637288 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216b267-a7a7-4b98-9117-9d48c102b68e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:46.637977 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216b267-a7a7-4b98-9117-9d48c102b68e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:46.637977 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216b267-a7a7-4b98-9117-9d48c102b68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:46.664538 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e216b267-a7a7-4b98-9117-9d48c102b68e None None] MFA Rules not processed for user `31c0594d0be84154a601d625bca699cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:33:46.665797 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e216b267-a7a7-4b98-9117-9d48c102b68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:46.796871 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1209/2401] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:33:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:33:46.801561 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90c81f-d5ff-4142-a0b5-09ab9abe81ae service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:46.811915 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90c81f-d5ff-4142-a0b5-09ab9abe81ae service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:46.817008 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac1597-eae8-4010-bd5a-4a2a9033198e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:46.817695 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac1597-eae8-4010-bd5a-4a2a9033198e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:46.817980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac1597-eae8-4010-bd5a-4a2a9033198e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:46.818271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac1597-eae8-4010-bd5a-4a2a9033198e admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:46.819059 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ac1597-eae8-4010-bd5a-4a2a9033198e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:46.821010 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ac1597-eae8-4010-bd5a-4a2a9033198e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:46.854458 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1193/2402] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:33:46 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:33:46.858130 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216b267-a7a7-4b98-9117-9d48c102b68e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:46.879001 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216b267-a7a7-4b98-9117-9d48c102b68e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:47.020116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe7a4f-22ab-46ed-9dcf-10aedfee4f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:47.020791 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe7a4f-22ab-46ed-9dcf-10aedfee4f58 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:47.021080 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe7a4f-22ab-46ed-9dcf-10aedfee4f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:47.021462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe7a4f-22ab-46ed-9dcf-10aedfee4f58 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:47.022230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe7a4f-22ab-46ed-9dcf-10aedfee4f58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:47.024187 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe7a4f-22ab-46ed-9dcf-10aedfee4f58 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:47.063199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1210/2403] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:33:46 2022] POST /identity/v3/users => generated 296 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:33:47.067494 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac1597-eae8-4010-bd5a-4a2a9033198e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:47.078694 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac1597-eae8-4010-bd5a-4a2a9033198e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:47.099514 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc45331-66b1-42fe-9b5d-0a1f839ba4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:47.100345 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc45331-66b1-42fe-9b5d-0a1f839ba4d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:47.100705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc45331-66b1-42fe-9b5d-0a1f839ba4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:47.101059 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc45331-66b1-42fe-9b5d-0a1f839ba4d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:47.101999 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc45331-66b1-42fe-9b5d-0a1f839ba4d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:47.103764 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc45331-66b1-42fe-9b5d-0a1f839ba4d2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:47.122293 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1194/2404] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:33:47.126477 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe7a4f-22ab-46ed-9dcf-10aedfee4f58 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:47.141052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe7a4f-22ab-46ed-9dcf-10aedfee4f58 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:47.148390 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56451262-17ee-446a-b606-b19c250bbd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:47.150092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56451262-17ee-446a-b606-b19c250bbd37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:47.150664 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56451262-17ee-446a-b606-b19c250bbd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:47.151333 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56451262-17ee-446a-b606-b19c250bbd37 admin admin] PATH_INFO: `/v3/projects/7571f5ecf2c34afe98111f44580632f8/users/4b80bd5d4c674b6c98cb91b1de7bf01b/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:47.172972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56451262-17ee-446a-b606-b19c250bbd37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7571f5ecf2c34afe98111f44580632f8, user_id=4b80bd5d4c674b6c98cb91b1de7bf01b, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:47.176308 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56451262-17ee-446a-b606-b19c250bbd37 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:47.198131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1211/2405] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:33:47 2022] PUT /identity/v3/projects/7571f5ecf2c34afe98111f44580632f8/users/4b80bd5d4c674b6c98cb91b1de7bf01b/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:33:47.202611 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc45331-66b1-42fe-9b5d-0a1f839ba4d2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:47.215093 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc45331-66b1-42fe-9b5d-0a1f839ba4d2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:47.292659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3a817-fc89-4ccf-b465-83fa90c690eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:47.293534 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3a817-fc89-4ccf-b465-83fa90c690eb admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:47.294054 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3a817-fc89-4ccf-b465-83fa90c690eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:47.294340 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3a817-fc89-4ccf-b465-83fa90c690eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:47.295081 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a3a817-fc89-4ccf-b465-83fa90c690eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:47.296855 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a3a817-fc89-4ccf-b465-83fa90c690eb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:47.316048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1195/2406] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:33:47.319802 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56451262-17ee-446a-b606-b19c250bbd37 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:47.335787 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56451262-17ee-446a-b606-b19c250bbd37 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:47.336927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:33:47.342723 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c208a-cb0a-4736-b69f-705293b250c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:47.343795 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c208a-cb0a-4736-b69f-705293b250c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:47.344308 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c208a-cb0a-4736-b69f-705293b250c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:47.345136 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c208a-cb0a-4736-b69f-705293b250c9 admin admin] PATH_INFO: `/v3/projects/7571f5ecf2c34afe98111f44580632f8/users/4b80bd5d4c674b6c98cb91b1de7bf01b/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:47.348173 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c208a-cb0a-4736-b69f-705293b250c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7571f5ecf2c34afe98111f44580632f8, user_id=4b80bd5d4c674b6c98cb91b1de7bf01b, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:47.351284 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c208a-cb0a-4736-b69f-705293b250c9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:47.365651 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1212/2407] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:33:47 2022] PUT /identity/v3/projects/7571f5ecf2c34afe98111f44580632f8/users/4b80bd5d4c674b6c98cb91b1de7bf01b/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:33:53.319218 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a84d34-fa6f-4ace-a8ef-8efa0ba3e947 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:53.319834 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a84d34-fa6f-4ace-a8ef-8efa0ba3e947 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:53.319834 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a84d34-fa6f-4ace-a8ef-8efa0ba3e947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:53.347819 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-44a84d34-fa6f-4ace-a8ef-8efa0ba3e947 None None] MFA Rules not processed for user `4b80bd5d4c674b6c98cb91b1de7bf01b`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:33:53.349057 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-44a84d34-fa6f-4ace-a8ef-8efa0ba3e947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:53.398811 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:33:53.422578 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1196/2408] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:33:53 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:33:53.437297 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c208a-cb0a-4736-b69f-705293b250c9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:53.439523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0d2c208a-cb0a-4736-b69f-705293b250c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:53.519171 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c208a-cb0a-4736-b69f-705293b250c9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:53.528616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca437d1a-d19e-4dc0-93c2-09bdca4417ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:53.529628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca437d1a-d19e-4dc0-93c2-09bdca4417ba service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:53.530037 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca437d1a-d19e-4dc0-93c2-09bdca4417ba service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:53.530329 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca437d1a-d19e-4dc0-93c2-09bdca4417ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:53.591691 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca437d1a-d19e-4dc0-93c2-09bdca4417ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:53.593392 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca437d1a-d19e-4dc0-93c2-09bdca4417ba service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:53.602682 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1213/2409] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:33:53 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:34:03.419613 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:34:03.598550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=60,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:34:06.743957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a84d34-fa6f-4ace-a8ef-8efa0ba3e947 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:34:06.745547 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-44a84d34-fa6f-4ace-a8ef-8efa0ba3e947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:34:06.814943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a84d34-fa6f-4ace-a8ef-8efa0ba3e947 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:34:06.818605 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47970476-e8b4-48da-9cda-cc29b663bd07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:34:06.819250 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47970476-e8b4-48da-9cda-cc29b663bd07 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:34:06.819581 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47970476-e8b4-48da-9cda-cc29b663bd07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:34:06.819809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47970476-e8b4-48da-9cda-cc29b663bd07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:34:06.822041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-47970476-e8b4-48da-9cda-cc29b663bd07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:34:06.834326 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47970476-e8b4-48da-9cda-cc29b663bd07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:34:06.835780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47970476-e8b4-48da-9cda-cc29b663bd07 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:34:06.843920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1197/2410] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:34:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:34:09.225689 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cbb2a-a238-4e99-8f99-a0300b98fa64 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:34:09.226126 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cbb2a-a238-4e99-8f99-a0300b98fa64 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:34:09.226247 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cbb2a-a238-4e99-8f99-a0300b98fa64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:34:09.243306 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-728cbb2a-a238-4e99-8f99-a0300b98fa64 None None] MFA Rules not processed for user `4ddcf6306e704fe9ad5ed8779178ae67`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:34:09.244518 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-728cbb2a-a238-4e99-8f99-a0300b98fa64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:34:09.302692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1214/2411] 10.222.0.33 () {58 vars in 967 bytes} [Thu Jun 30 18:34:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:34:09.315937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47970476-e8b4-48da-9cda-cc29b663bd07 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:34:09.317329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-47970476-e8b4-48da-9cda-cc29b663bd07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:34:09.385590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47970476-e8b4-48da-9cda-cc29b663bd07 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:34:09.391301 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5716a-52cc-4c4f-b964-cf1951e6030c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bd521b9e93f4b3b8af64d46f36c019e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:34:09.391937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5716a-52cc-4c4f-b964-cf1951e6030c service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:34:09.392226 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5716a-52cc-4c4f-b964-cf1951e6030c service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:34:09.392504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5716a-52cc-4c4f-b964-cf1951e6030c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:34:09.400789 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5716a-52cc-4c4f-b964-cf1951e6030c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:34:09.402434 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5716a-52cc-4c4f-b964-cf1951e6030c service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:34:09.410781 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1198/2412] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 30 18:34:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 18:34:18.369023 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5efec2-08f6-4b62-b0ee-a5ae4bd49692 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:34:18.369640 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5efec2-08f6-4b62-b0ee-a5ae4bd49692 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:34:18.369640 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5efec2-08f6-4b62-b0ee-a5ae4bd49692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:34:18.388043 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fc5efec2-08f6-4b62-b0ee-a5ae4bd49692 None None] MFA Rules not processed for user `31c0594d0be84154a601d625bca699cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:34:18.390508 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fc5efec2-08f6-4b62-b0ee-a5ae4bd49692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:34:18.397430 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1215/2413] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:34:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:34:19.407576 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=58 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:34:21.899107 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5716a-52cc-4c4f-b964-cf1951e6030c service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:34:21.900681 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-80c5716a-52cc-4c4f-b964-cf1951e6030c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:34:21.918567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5716a-52cc-4c4f-b964-cf1951e6030c service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:34:21.924634 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f0799-77d9-4891-a73f-9f72ecf1c722 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:34:21.925292 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f0799-77d9-4891-a73f-9f72ecf1c722 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:34:21.925608 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f0799-77d9-4891-a73f-9f72ecf1c722 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:34:21.925905 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f0799-77d9-4891-a73f-9f72ecf1c722 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:34:21.927832 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b24f0799-77d9-4891-a73f-9f72ecf1c722 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:34:21.942153 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f0799-77d9-4891-a73f-9f72ecf1c722 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:34:21.943809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f0799-77d9-4891-a73f-9f72ecf1c722 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:34:21.957415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1199/2414] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:34:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:34:26.385032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5efec2-08f6-4b62-b0ee-a5ae4bd49692 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:34:26.398367 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5efec2-08f6-4b62-b0ee-a5ae4bd49692 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:34:26.403815 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ab9d9-8b53-4bab-8c2e-e8d5d5f69b35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:34:26.404599 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ab9d9-8b53-4bab-8c2e-e8d5d5f69b35 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:34:26.404885 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ab9d9-8b53-4bab-8c2e-e8d5d5f69b35 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:34:26.405166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ab9d9-8b53-4bab-8c2e-e8d5d5f69b35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:34:26.407022 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-df7ab9d9-8b53-4bab-8c2e-e8d5d5f69b35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:34:26.465922 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ab9d9-8b53-4bab-8c2e-e8d5d5f69b35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:34:26.467600 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ab9d9-8b53-4bab-8c2e-e8d5d5f69b35 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:34:26.477146 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1216/2415] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:34:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:34:31.952674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:34:34.290423 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f0799-77d9-4891-a73f-9f72ecf1c722 service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:34:34.306123 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f0799-77d9-4891-a73f-9f72ecf1c722 service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:34:34.313856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb8cff-8af7-46df-bc40-81f845b71edf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:34:34.314946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb8cff-8af7-46df-bc40-81f845b71edf service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:34:34.315335 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb8cff-8af7-46df-bc40-81f845b71edf service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:34:34.315938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb8cff-8af7-46df-bc40-81f845b71edf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:34:34.318342 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d0fb8cff-8af7-46df-bc40-81f845b71edf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:34:34.332706 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb8cff-8af7-46df-bc40-81f845b71edf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:34:34.334643 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb8cff-8af7-46df-bc40-81f845b71edf service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:34:34.360583 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1200/2416] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:34:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:34:36.472502 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=56 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:34:37.400241 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ab9d9-8b53-4bab-8c2e-e8d5d5f69b35 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:34:37.410366 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ab9d9-8b53-4bab-8c2e-e8d5d5f69b35 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:34:37.413950 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88668241-c0e5-4a17-b9f9-4c9019fc6e7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:34:37.414584 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88668241-c0e5-4a17-b9f9-4c9019fc6e7b service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:34:37.414866 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88668241-c0e5-4a17-b9f9-4c9019fc6e7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:34:37.415144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88668241-c0e5-4a17-b9f9-4c9019fc6e7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:34:37.424842 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88668241-c0e5-4a17-b9f9-4c9019fc6e7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:34:37.426346 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88668241-c0e5-4a17-b9f9-4c9019fc6e7b service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:34:37.435573 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1217/2417] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:34:37 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:34:39.051396 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb8cff-8af7-46df-bc40-81f845b71edf service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:34:39.063240 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb8cff-8af7-46df-bc40-81f845b71edf service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:34:39.068880 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799ca70-d594-46c2-87cc-61bf747c26d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:34:39.069465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799ca70-d594-46c2-87cc-61bf747c26d0 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:34:39.069779 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799ca70-d594-46c2-87cc-61bf747c26d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:34:39.070055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799ca70-d594-46c2-87cc-61bf747c26d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:34:39.071835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0799ca70-d594-46c2-87cc-61bf747c26d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:34:39.086472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799ca70-d594-46c2-87cc-61bf747c26d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:34:39.088193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799ca70-d594-46c2-87cc-61bf747c26d0 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:34:39.101165 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1201/2418] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:34:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:34:47.431413 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:34:49.095763 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=12 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:35:01.692373 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b35a13-9545-4da3-8abe-4b6948c7c28c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:35:01.692883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b35a13-9545-4da3-8abe-4b6948c7c28c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:35:01.692984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b35a13-9545-4da3-8abe-4b6948c7c28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:35:01.710074 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d8b35a13-9545-4da3-8abe-4b6948c7c28c None None] MFA Rules not processed for user `c66efdb1cc27467d85831617590c8b28`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:35:01.711262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d8b35a13-9545-4da3-8abe-4b6948c7c28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:35:01.718114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1218/2419] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:35:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:35:11.705044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:35:25.307745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea3fd4-5ad9-47cd-aade-d2a58462e0e4 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:35:25.308275 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea3fd4-5ad9-47cd-aade-d2a58462e0e4 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:35:25.308523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea3fd4-5ad9-47cd-aade-d2a58462e0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:35:25.326773 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fbea3fd4-5ad9-47cd-aade-d2a58462e0e4 None None] MFA Rules not processed for user `f13e9e1bf5084d08919845767c70c3f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:35:25.327968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fbea3fd4-5ad9-47cd-aade-d2a58462e0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:35:25.392411 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1202/2420] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:35:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:35:26.691007 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b35a13-9545-4da3-8abe-4b6948c7c28c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:35:26.703342 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b35a13-9545-4da3-8abe-4b6948c7c28c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:35:26.709212 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac385f-7a21-41e2-933e-924ffd3f30bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:35:26.709897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac385f-7a21-41e2-933e-924ffd3f30bb service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:35:26.710181 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac385f-7a21-41e2-933e-924ffd3f30bb service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:35:26.710457 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac385f-7a21-41e2-933e-924ffd3f30bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:35:26.712203 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-59ac385f-7a21-41e2-933e-924ffd3f30bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:35:26.769207 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac385f-7a21-41e2-933e-924ffd3f30bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:35:26.770811 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac385f-7a21-41e2-933e-924ffd3f30bb service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:35:26.779667 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1219/2421] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:35:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:35:35.385632 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=25 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:35:36.775508 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=28 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:17.147433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea3fd4-5ad9-47cd-aade-d2a58462e0e4 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:17.156898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea3fd4-5ad9-47cd-aade-d2a58462e0e4 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:17.160475 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa436c41-e9ee-4cd7-b9a3-0f844d06e078 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:17.161112 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa436c41-e9ee-4cd7-b9a3-0f844d06e078 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:17.161398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa436c41-e9ee-4cd7-b9a3-0f844d06e078 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:17.161722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa436c41-e9ee-4cd7-b9a3-0f844d06e078 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:17.164185 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fa436c41-e9ee-4cd7-b9a3-0f844d06e078 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:17.227033 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa436c41-e9ee-4cd7-b9a3-0f844d06e078 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:17.228485 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa436c41-e9ee-4cd7-b9a3-0f844d06e078 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:17.238113 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1203/2422] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:36:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:36:20.388309 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac385f-7a21-41e2-933e-924ffd3f30bb service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:20.390141 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-59ac385f-7a21-41e2-933e-924ffd3f30bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:20.415116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac385f-7a21-41e2-933e-924ffd3f30bb service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:20.420038 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4442916-539e-41ce-a160-a864208d04a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:20.420922 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4442916-539e-41ce-a160-a864208d04a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:20.421210 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4442916-539e-41ce-a160-a864208d04a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:20.421491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4442916-539e-41ce-a160-a864208d04a2 admin admin] PATH_INFO: `/v3/users/c66efdb1cc27467d85831617590c8b28` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:20.422795 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4442916-539e-41ce-a160-a864208d04a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c66efdb1cc27467d85831617590c8b28)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:20.424740 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4442916-539e-41ce-a160-a864208d04a2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:20.475597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d4442916-539e-41ce-a160-a864208d04a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c66efdb1cc27467d85831617590c8b28'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:20.487135 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d4442916-539e-41ce-a160-a864208d04a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c66efdb1cc27467d85831617590c8b28'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:20.487806 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d4442916-539e-41ce-a160-a864208d04a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c66efdb1cc27467d85831617590c8b28'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:20.494711 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d4442916-539e-41ce-a160-a864208d04a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c66efdb1cc27467d85831617590c8b28'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:20.495365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d4442916-539e-41ce-a160-a864208d04a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c66efdb1cc27467d85831617590c8b28'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:20.508518 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1220/2423] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:36:20 2022] DELETE /identity/v3/users/c66efdb1cc27467d85831617590c8b28 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:20.677365 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa436c41-e9ee-4cd7-b9a3-0f844d06e078 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:20.688638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa436c41-e9ee-4cd7-b9a3-0f844d06e078 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:20.744583 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2dea15-0c43-4c4c-a0b9-b9b8e81c21ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:20.745434 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dea15-0c43-4c4c-a0b9-b9b8e81c21ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:20.745753 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dea15-0c43-4c4c-a0b9-b9b8e81c21ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:20.746033 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dea15-0c43-4c4c-a0b9-b9b8e81c21ec admin admin] PATH_INFO: `/v3/projects/89156f1867314be89e5108d5506b83a3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:20.747192 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2dea15-0c43-4c4c-a0b9-b9b8e81c21ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=89156f1867314be89e5108d5506b83a3)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:20.749096 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2dea15-0c43-4c4c-a0b9-b9b8e81c21ec admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:20.793788 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1d2dea15-0c43-4c4c-a0b9-b9b8e81c21ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89156f1867314be89e5108d5506b83a3', 'request_id': 'req-1d2dea15-0c43-4c4c-a0b9-b9b8e81c21ec'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:20.799676 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1d2dea15-0c43-4c4c-a0b9-b9b8e81c21ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89156f1867314be89e5108d5506b83a3', 'request_id': 'req-1d2dea15-0c43-4c4c-a0b9-b9b8e81c21ec'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:20.805664 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1d2dea15-0c43-4c4c-a0b9-b9b8e81c21ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89156f1867314be89e5108d5506b83a3', 'request_id': 'req-1d2dea15-0c43-4c4c-a0b9-b9b8e81c21ec'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:20.818312 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1d2dea15-0c43-4c4c-a0b9-b9b8e81c21ec admin admin] The token cache is being invalidate because project 89156f1867314be89e5108d5506b83a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:36:20.819109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1d2dea15-0c43-4c4c-a0b9-b9b8e81c21ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:20.820925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1d2dea15-0c43-4c4c-a0b9-b9b8e81c21ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:20.822685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1204/2424] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:36:20 2022] DELETE /identity/v3/projects/89156f1867314be89e5108d5506b83a3 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:20.827887 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e58a5-c5af-45cf-8966-18084151b627 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:20.828183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e58a5-c5af-45cf-8966-18084151b627 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:20.828477 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e58a5-c5af-45cf-8966-18084151b627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:20.846543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d56e58a5-c5af-45cf-8966-18084151b627 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:36:20.847759 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d56e58a5-c5af-45cf-8966-18084151b627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:20.908866 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1221/2425] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:36:20.918123 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406c79f-afdb-47f1-a706-f40695ea0303 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:20.918418 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406c79f-afdb-47f1-a706-f40695ea0303 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:20.918700 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406c79f-afdb-47f1-a706-f40695ea0303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:20.938804 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1406c79f-afdb-47f1-a706-f40695ea0303 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:36:20.940111 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1406c79f-afdb-47f1-a706-f40695ea0303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:20.948902 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1205/2426] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:36:20.953159 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e58a5-c5af-45cf-8966-18084151b627 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:20.963225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e58a5-c5af-45cf-8966-18084151b627 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:20.967915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b3610-1c75-4bcb-b3c7-4ff46b9094c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:20.968576 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b3610-1c75-4bcb-b3c7-4ff46b9094c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:20.968867 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b3610-1c75-4bcb-b3c7-4ff46b9094c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:20.969150 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b3610-1c75-4bcb-b3c7-4ff46b9094c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:20.969914 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b3610-1c75-4bcb-b3c7-4ff46b9094c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:36:20.970225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b3610-1c75-4bcb-b3c7-4ff46b9094c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:20.971840 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b3610-1c75-4bcb-b3c7-4ff46b9094c6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:20.987715 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1222/2427] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:36:20.988799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406c79f-afdb-47f1-a706-f40695ea0303 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:20.998867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406c79f-afdb-47f1-a706-f40695ea0303 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:21.003799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d41d2f-baa8-4e7a-8395-081566867d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:21.004447 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d41d2f-baa8-4e7a-8395-081566867d96 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:21.004727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d41d2f-baa8-4e7a-8395-081566867d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:21.005004 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d41d2f-baa8-4e7a-8395-081566867d96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:21.005723 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d41d2f-baa8-4e7a-8395-081566867d96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:21.008308 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d41d2f-baa8-4e7a-8395-081566867d96 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:21.023250 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:21.048432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1206/2428] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:36:20 2022] POST /identity/v3/projects => generated 405 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:36:21.052565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b3610-1c75-4bcb-b3c7-4ff46b9094c6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:21.063102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b3610-1c75-4bcb-b3c7-4ff46b9094c6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:21.127753 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e25bb-4183-4930-b717-c829edbce496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:21.128436 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e25bb-4183-4930-b717-c829edbce496 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:21.128797 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e25bb-4183-4930-b717-c829edbce496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:21.129150 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e25bb-4183-4930-b717-c829edbce496 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:21.130004 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e25bb-4183-4930-b717-c829edbce496 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:21.132584 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e25bb-4183-4930-b717-c829edbce496 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:21.139626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:21.156326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1223/2429] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:36:21 2022] POST /identity/v3/users => generated 303 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:36:21.160473 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d41d2f-baa8-4e7a-8395-081566867d96 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:21.172142 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d41d2f-baa8-4e7a-8395-081566867d96 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:21.177728 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af7070-6fcf-40aa-bd5b-d87b6e7c6b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:21.178466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af7070-6fcf-40aa-bd5b-d87b6e7c6b8c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:21.178898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af7070-6fcf-40aa-bd5b-d87b6e7c6b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:21.179392 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af7070-6fcf-40aa-bd5b-d87b6e7c6b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:21.180321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af7070-6fcf-40aa-bd5b-d87b6e7c6b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:21.182143 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af7070-6fcf-40aa-bd5b-d87b6e7c6b8c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:21.193304 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1207/2430] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:36:21.198699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e25bb-4183-4930-b717-c829edbce496 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:21.208986 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e25bb-4183-4930-b717-c829edbce496 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:21.214804 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024158ba-8a52-4b8a-9084-57cc83960423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:21.215938 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024158ba-8a52-4b8a-9084-57cc83960423 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:21.216358 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024158ba-8a52-4b8a-9084-57cc83960423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:21.216754 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024158ba-8a52-4b8a-9084-57cc83960423 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:21.217637 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024158ba-8a52-4b8a-9084-57cc83960423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:21.219361 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024158ba-8a52-4b8a-9084-57cc83960423 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:21.230611 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1224/2431] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:36:21.234674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af7070-6fcf-40aa-bd5b-d87b6e7c6b8c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:21.245025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af7070-6fcf-40aa-bd5b-d87b6e7c6b8c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:21.249759 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e821a5-00be-40ad-b603-70da2d6eff23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:21.250633 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e821a5-00be-40ad-b603-70da2d6eff23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:21.250918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e821a5-00be-40ad-b603-70da2d6eff23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:21.251221 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e821a5-00be-40ad-b603-70da2d6eff23 admin admin] PATH_INFO: `/v3/projects/c68678053a8b4792bcd6974c341bec06/users/df9ed26eb89b4f54a26d68a0667551ef/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:21.267469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e821a5-00be-40ad-b603-70da2d6eff23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c68678053a8b4792bcd6974c341bec06, user_id=df9ed26eb89b4f54a26d68a0667551ef, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:21.270457 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e821a5-00be-40ad-b603-70da2d6eff23 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:21.285684 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1208/2432] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:36:21 2022] PUT /identity/v3/projects/c68678053a8b4792bcd6974c341bec06/users/df9ed26eb89b4f54a26d68a0667551ef/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:21.296307 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1183795-d9f0-4d64-a178-56ffc9cbd3f2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:21.296603 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1183795-d9f0-4d64-a178-56ffc9cbd3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:21.296888 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1183795-d9f0-4d64-a178-56ffc9cbd3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:21.326604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b1183795-d9f0-4d64-a178-56ffc9cbd3f2 None None] MFA Rules not processed for user `df9ed26eb89b4f54a26d68a0667551ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:36:21.327824 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b1183795-d9f0-4d64-a178-56ffc9cbd3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:21.394029 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1225/2433] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:36:21 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:36:21.398567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e821a5-00be-40ad-b603-70da2d6eff23 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:21.458733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e821a5-00be-40ad-b603-70da2d6eff23 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:21.463628 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708d223-81ca-4afb-9ab0-093d972aa36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:21.464263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708d223-81ca-4afb-9ab0-093d972aa36e admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:21.464544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708d223-81ca-4afb-9ab0-093d972aa36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:21.464822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708d223-81ca-4afb-9ab0-093d972aa36e admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:21.465502 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708d223-81ca-4afb-9ab0-093d972aa36e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:21.467429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708d223-81ca-4afb-9ab0-093d972aa36e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:21.491864 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1209/2434] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:36:21 2022] POST /identity/v3/projects => generated 403 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:36:21.496332 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1183795-d9f0-4d64-a178-56ffc9cbd3f2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:21.507500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1183795-d9f0-4d64-a178-56ffc9cbd3f2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:21.563882 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5234b-15a6-4910-8a2c-9f578332f9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:21.564545 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5234b-15a6-4910-8a2c-9f578332f9f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:21.564858 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5234b-15a6-4910-8a2c-9f578332f9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:21.565138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5234b-15a6-4910-8a2c-9f578332f9f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:21.565863 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5234b-15a6-4910-8a2c-9f578332f9f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:21.567779 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5234b-15a6-4910-8a2c-9f578332f9f8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:21.587375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1226/2435] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:36:21 2022] POST /identity/v3/users => generated 308 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:36:21.591320 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708d223-81ca-4afb-9ab0-093d972aa36e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:21.601907 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708d223-81ca-4afb-9ab0-093d972aa36e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:21.606430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867b0db-b5c7-40d3-9920-ff364f5af599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:21.607069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867b0db-b5c7-40d3-9920-ff364f5af599 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:21.607349 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867b0db-b5c7-40d3-9920-ff364f5af599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:21.607623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867b0db-b5c7-40d3-9920-ff364f5af599 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:21.608364 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867b0db-b5c7-40d3-9920-ff364f5af599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:21.610006 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867b0db-b5c7-40d3-9920-ff364f5af599 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:21.620653 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1210/2436] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:36:21.624582 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5234b-15a6-4910-8a2c-9f578332f9f8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:21.635843 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5234b-15a6-4910-8a2c-9f578332f9f8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:21.640498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c64a0-dade-419a-b974-7fa4badfad11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:21.641367 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c64a0-dade-419a-b974-7fa4badfad11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:21.641693 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c64a0-dade-419a-b974-7fa4badfad11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:21.641974 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c64a0-dade-419a-b974-7fa4badfad11 admin admin] PATH_INFO: `/v3/projects/611138ee926c4937beda79627c7ee95b/users/134a869759dd43ca82f9e4a15fbace8c/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:21.658830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c64a0-dade-419a-b974-7fa4badfad11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=611138ee926c4937beda79627c7ee95b, user_id=134a869759dd43ca82f9e4a15fbace8c, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:21.661433 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c64a0-dade-419a-b974-7fa4badfad11 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:21.674830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1227/2437] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:36:21 2022] PUT /identity/v3/projects/611138ee926c4937beda79627c7ee95b/users/134a869759dd43ca82f9e4a15fbace8c/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:21.678757 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867b0db-b5c7-40d3-9920-ff364f5af599 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:21.689544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867b0db-b5c7-40d3-9920-ff364f5af599 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:21.745724 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed98c3a-3ceb-472b-8f18-04653fb1895b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:21.746380 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed98c3a-3ceb-472b-8f18-04653fb1895b admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:21.746668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed98c3a-3ceb-472b-8f18-04653fb1895b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:21.746949 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed98c3a-3ceb-472b-8f18-04653fb1895b admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:21.747671 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed98c3a-3ceb-472b-8f18-04653fb1895b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:21.749264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed98c3a-3ceb-472b-8f18-04653fb1895b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:21.759718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1211/2438] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:36:21.763842 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c64a0-dade-419a-b974-7fa4badfad11 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:21.774826 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c64a0-dade-419a-b974-7fa4badfad11 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:21.779998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58840b7f-fd73-40d9-83bf-3a693b5310e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:21.780881 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58840b7f-fd73-40d9-83bf-3a693b5310e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:21.781172 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58840b7f-fd73-40d9-83bf-3a693b5310e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:21.781480 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58840b7f-fd73-40d9-83bf-3a693b5310e0 admin admin] PATH_INFO: `/v3/projects/611138ee926c4937beda79627c7ee95b/users/134a869759dd43ca82f9e4a15fbace8c/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:21.783477 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58840b7f-fd73-40d9-83bf-3a693b5310e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=611138ee926c4937beda79627c7ee95b, user_id=134a869759dd43ca82f9e4a15fbace8c, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:21.786181 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58840b7f-fd73-40d9-83bf-3a693b5310e0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:21.796414 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1228/2439] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:36:21 2022] PUT /identity/v3/projects/611138ee926c4937beda79627c7ee95b/users/134a869759dd43ca82f9e4a15fbace8c/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:21.806401 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bfb00-5af6-4a73-b4e7-3f2ba8948b4d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:21.806696 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bfb00-5af6-4a73-b4e7-3f2ba8948b4d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:21.806979 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bfb00-5af6-4a73-b4e7-3f2ba8948b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:21.832994 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-514bfb00-5af6-4a73-b4e7-3f2ba8948b4d None None] MFA Rules not processed for user `134a869759dd43ca82f9e4a15fbace8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:36:21.834240 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-514bfb00-5af6-4a73-b4e7-3f2ba8948b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:21.846963 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:21.907365 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1212/2440] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:36:21 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:36:21.918495 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58840b7f-fd73-40d9-83bf-3a693b5310e0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:21.920175 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-58840b7f-fd73-40d9-83bf-3a693b5310e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:21.941937 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:21.977913 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58840b7f-fd73-40d9-83bf-3a693b5310e0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:21.981492 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eda7d7-b649-4d98-aa1e-dd43c865ed77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:21.982154 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eda7d7-b649-4d98-aa1e-dd43c865ed77 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:21.982438 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eda7d7-b649-4d98-aa1e-dd43c865ed77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:21.982715 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eda7d7-b649-4d98-aa1e-dd43c865ed77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:21.991541 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eda7d7-b649-4d98-aa1e-dd43c865ed77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:21.992947 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eda7d7-b649-4d98-aa1e-dd43c865ed77 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:22.002144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1229/2441] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:36:21 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:36:22.147741 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514bfb00-5af6-4a73-b4e7-3f2ba8948b4d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:22.207543 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514bfb00-5af6-4a73-b4e7-3f2ba8948b4d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:22.213024 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb9ddd-8917-4229-99d1-c9906c73a3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:22.213906 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb9ddd-8917-4229-99d1-c9906c73a3ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:22.214186 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb9ddd-8917-4229-99d1-c9906c73a3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:22.214457 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb9ddd-8917-4229-99d1-c9906c73a3ea admin admin] PATH_INFO: `/v3/users/df9ed26eb89b4f54a26d68a0667551ef` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:22.215624 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb9ddd-8917-4229-99d1-c9906c73a3ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=df9ed26eb89b4f54a26d68a0667551ef)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:22.217554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb9ddd-8917-4229-99d1-c9906c73a3ea admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:22.263176 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6ddb9ddd-8917-4229-99d1-c9906c73a3ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df9ed26eb89b4f54a26d68a0667551ef'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:22.263859 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6ddb9ddd-8917-4229-99d1-c9906c73a3ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df9ed26eb89b4f54a26d68a0667551ef'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:22.276827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6ddb9ddd-8917-4229-99d1-c9906c73a3ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df9ed26eb89b4f54a26d68a0667551ef'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:22.284002 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6ddb9ddd-8917-4229-99d1-c9906c73a3ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df9ed26eb89b4f54a26d68a0667551ef'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:22.284604 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-6ddb9ddd-8917-4229-99d1-c9906c73a3ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df9ed26eb89b4f54a26d68a0667551ef'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:22.297315 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1213/2442] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:36:22 2022] DELETE /identity/v3/users/df9ed26eb89b4f54a26d68a0667551ef => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:22.301819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543c0ea-f07e-40c8-bacb-059c58076fb8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:22.302110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543c0ea-f07e-40c8-bacb-059c58076fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:22.302388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543c0ea-f07e-40c8-bacb-059c58076fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:22.319660 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-c543c0ea-f07e-40c8-bacb-059c58076fb8 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:36:22.320848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c543c0ea-f07e-40c8-bacb-059c58076fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:22.378691 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1230/2443] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:36:22.389233 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb9ddd-8917-4229-99d1-c9906c73a3ea admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:22.390651 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6ddb9ddd-8917-4229-99d1-c9906c73a3ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:22.422037 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:22.450546 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb9ddd-8917-4229-99d1-c9906c73a3ea admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:22.454233 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6cc85-6fdb-42b9-b2a2-1aa44035b8f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:22.454873 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6cc85-6fdb-42b9-b2a2-1aa44035b8f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:22.455159 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6cc85-6fdb-42b9-b2a2-1aa44035b8f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:22.455439 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6cc85-6fdb-42b9-b2a2-1aa44035b8f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:22.465397 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6cc85-6fdb-42b9-b2a2-1aa44035b8f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:22.466829 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6cc85-6fdb-42b9-b2a2-1aa44035b8f3 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:22.476449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1214/2444] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:36:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:36:22.746428 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c543c0ea-f07e-40c8-bacb-059c58076fb8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:22.748277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c543c0ea-f07e-40c8-bacb-059c58076fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:22.764065 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c543c0ea-f07e-40c8-bacb-059c58076fb8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:22.769210 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b61c8-208a-4cf3-a7f7-c9f89849248e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:22.770185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b61c8-208a-4cf3-a7f7-c9f89849248e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:22.770484 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b61c8-208a-4cf3-a7f7-c9f89849248e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:22.770767 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b61c8-208a-4cf3-a7f7-c9f89849248e admin admin] PATH_INFO: `/v3/projects/c68678053a8b4792bcd6974c341bec06` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:22.771966 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204b61c8-208a-4cf3-a7f7-c9f89849248e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c68678053a8b4792bcd6974c341bec06)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:22.774631 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204b61c8-208a-4cf3-a7f7-c9f89849248e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:22.821905 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-204b61c8-208a-4cf3-a7f7-c9f89849248e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c68678053a8b4792bcd6974c341bec06', 'request_id': 'req-204b61c8-208a-4cf3-a7f7-c9f89849248e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:22.827999 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-204b61c8-208a-4cf3-a7f7-c9f89849248e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c68678053a8b4792bcd6974c341bec06', 'request_id': 'req-204b61c8-208a-4cf3-a7f7-c9f89849248e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:22.833453 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-204b61c8-208a-4cf3-a7f7-c9f89849248e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c68678053a8b4792bcd6974c341bec06', 'request_id': 'req-204b61c8-208a-4cf3-a7f7-c9f89849248e'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:22.845565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-204b61c8-208a-4cf3-a7f7-c9f89849248e admin admin] The token cache is being invalidate because project c68678053a8b4792bcd6974c341bec06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:36:22.845896 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-204b61c8-208a-4cf3-a7f7-c9f89849248e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:22.846545 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-204b61c8-208a-4cf3-a7f7-c9f89849248e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:22.848062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1231/2445] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:36:22 2022] DELETE /identity/v3/projects/c68678053a8b4792bcd6974c341bec06 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:22.852080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6cc85-6fdb-42b9-b2a2-1aa44035b8f3 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:22.861666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6cc85-6fdb-42b9-b2a2-1aa44035b8f3 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:22.918378 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de5e91-66ce-4c9b-a9e3-7826431bd562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:22.919240 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de5e91-66ce-4c9b-a9e3-7826431bd562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:22.919525 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de5e91-66ce-4c9b-a9e3-7826431bd562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:22.919801 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de5e91-66ce-4c9b-a9e3-7826431bd562 admin admin] PATH_INFO: `/v3/users/134a869759dd43ca82f9e4a15fbace8c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:22.920975 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de5e91-66ce-4c9b-a9e3-7826431bd562 admin admin] RBAC: Authorizing `identity:delete_user(user_id=134a869759dd43ca82f9e4a15fbace8c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:22.922935 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de5e91-66ce-4c9b-a9e3-7826431bd562 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:22.968954 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-35de5e91-66ce-4c9b-a9e3-7826431bd562 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '134a869759dd43ca82f9e4a15fbace8c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:22.969677 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-35de5e91-66ce-4c9b-a9e3-7826431bd562 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '134a869759dd43ca82f9e4a15fbace8c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:22.981905 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-35de5e91-66ce-4c9b-a9e3-7826431bd562 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '134a869759dd43ca82f9e4a15fbace8c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:22.989202 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-35de5e91-66ce-4c9b-a9e3-7826431bd562 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '134a869759dd43ca82f9e4a15fbace8c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:22.989842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-35de5e91-66ce-4c9b-a9e3-7826431bd562 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '134a869759dd43ca82f9e4a15fbace8c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:23.003110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1215/2446] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:36:22 2022] DELETE /identity/v3/users/134a869759dd43ca82f9e4a15fbace8c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:23.278841 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b61c8-208a-4cf3-a7f7-c9f89849248e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:23.280312 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-204b61c8-208a-4cf3-a7f7-c9f89849248e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:23.293513 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b61c8-208a-4cf3-a7f7-c9f89849248e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:23.344558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:23.358889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab1a6c-3aeb-49da-985a-adf46b494c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:23.359764 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab1a6c-3aeb-49da-985a-adf46b494c5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:23.360049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab1a6c-3aeb-49da-985a-adf46b494c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:23.360346 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab1a6c-3aeb-49da-985a-adf46b494c5a admin admin] PATH_INFO: `/v3/projects/611138ee926c4937beda79627c7ee95b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:23.361519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab1a6c-3aeb-49da-985a-adf46b494c5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=611138ee926c4937beda79627c7ee95b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:23.363491 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab1a6c-3aeb-49da-985a-adf46b494c5a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:23.410051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-54ab1a6c-3aeb-49da-985a-adf46b494c5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '611138ee926c4937beda79627c7ee95b', 'request_id': 'req-54ab1a6c-3aeb-49da-985a-adf46b494c5a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:23.416657 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-54ab1a6c-3aeb-49da-985a-adf46b494c5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '611138ee926c4937beda79627c7ee95b', 'request_id': 'req-54ab1a6c-3aeb-49da-985a-adf46b494c5a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:23.421994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-54ab1a6c-3aeb-49da-985a-adf46b494c5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '611138ee926c4937beda79627c7ee95b', 'request_id': 'req-54ab1a6c-3aeb-49da-985a-adf46b494c5a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:23.434452 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-54ab1a6c-3aeb-49da-985a-adf46b494c5a admin admin] The token cache is being invalidate because project 611138ee926c4937beda79627c7ee95b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:36:23.434741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-54ab1a6c-3aeb-49da-985a-adf46b494c5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:23.435354 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-54ab1a6c-3aeb-49da-985a-adf46b494c5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:23.436866 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1232/2447] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:36:23 2022] DELETE /identity/v3/projects/611138ee926c4937beda79627c7ee95b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:23.441707 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92544850-0c15-4abf-8636-43f5b0bf25fe None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:23.442000 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92544850-0c15-4abf-8636-43f5b0bf25fe None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:23.442282 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92544850-0c15-4abf-8636-43f5b0bf25fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:23.462409 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-92544850-0c15-4abf-8636-43f5b0bf25fe None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:36:23.463614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-92544850-0c15-4abf-8636-43f5b0bf25fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:23.508815 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:23.533224 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1216/2448] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:36:23.541442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6e2cd-de57-4e2a-8f6b-89a5e04be795 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:23.541780 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6e2cd-de57-4e2a-8f6b-89a5e04be795 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:23.542063 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6e2cd-de57-4e2a-8f6b-89a5e04be795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:23.560183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b1c6e2cd-de57-4e2a-8f6b-89a5e04be795 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:36:23.561382 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b1c6e2cd-de57-4e2a-8f6b-89a5e04be795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:23.569208 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1233/2449] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:36:23.573565 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92544850-0c15-4abf-8636-43f5b0bf25fe None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:23.583366 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92544850-0c15-4abf-8636-43f5b0bf25fe None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:23.589042 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37336d95-bc1d-45d8-9fe9-9cc5f602ecf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:23.589725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37336d95-bc1d-45d8-9fe9-9cc5f602ecf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:23.590014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37336d95-bc1d-45d8-9fe9-9cc5f602ecf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:23.590306 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37336d95-bc1d-45d8-9fe9-9cc5f602ecf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:23.591026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37336d95-bc1d-45d8-9fe9-9cc5f602ecf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:36:23.591333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37336d95-bc1d-45d8-9fe9-9cc5f602ecf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:23.592936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37336d95-bc1d-45d8-9fe9-9cc5f602ecf8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:23.605085 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1217/2450] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:36:23.609267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6e2cd-de57-4e2a-8f6b-89a5e04be795 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:23.619290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6e2cd-de57-4e2a-8f6b-89a5e04be795 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:23.624108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4d775-fb48-4c06-adfc-67616217b2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:23.624755 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4d775-fb48-4c06-adfc-67616217b2f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:23.625039 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4d775-fb48-4c06-adfc-67616217b2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:23.625317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4d775-fb48-4c06-adfc-67616217b2f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:23.626039 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4d775-fb48-4c06-adfc-67616217b2f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:23.627932 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4d775-fb48-4c06-adfc-67616217b2f7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:23.657988 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1234/2451] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:36:23 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:36:23.662077 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37336d95-bc1d-45d8-9fe9-9cc5f602ecf8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:23.672732 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37336d95-bc1d-45d8-9fe9-9cc5f602ecf8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:23.736683 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4357d8-e33d-4023-98b7-2ab31e107591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:23.737377 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4357d8-e33d-4023-98b7-2ab31e107591 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:23.737693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4357d8-e33d-4023-98b7-2ab31e107591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:23.737972 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4357d8-e33d-4023-98b7-2ab31e107591 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:23.738669 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4357d8-e33d-4023-98b7-2ab31e107591 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:23.740595 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4357d8-e33d-4023-98b7-2ab31e107591 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:23.761177 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1218/2452] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:36:23 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:36:23.765421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4d775-fb48-4c06-adfc-67616217b2f7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:23.776226 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4d775-fb48-4c06-adfc-67616217b2f7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:23.780894 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30b809-ed52-48df-a0b4-35587b4f8f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:23.781572 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30b809-ed52-48df-a0b4-35587b4f8f3f admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:23.781889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30b809-ed52-48df-a0b4-35587b4f8f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:23.782170 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30b809-ed52-48df-a0b4-35587b4f8f3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:23.782896 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db30b809-ed52-48df-a0b4-35587b4f8f3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:23.784547 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db30b809-ed52-48df-a0b4-35587b4f8f3f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:23.794673 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1235/2453] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:36:23.798438 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4357d8-e33d-4023-98b7-2ab31e107591 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:23.808766 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4357d8-e33d-4023-98b7-2ab31e107591 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:23.813417 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8ecf6-a217-4753-87e7-74fde4750806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:23.814101 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8ecf6-a217-4753-87e7-74fde4750806 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:23.814386 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8ecf6-a217-4753-87e7-74fde4750806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:23.814667 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8ecf6-a217-4753-87e7-74fde4750806 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:23.815389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb8ecf6-a217-4753-87e7-74fde4750806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:23.816977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb8ecf6-a217-4753-87e7-74fde4750806 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:23.827333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1219/2454] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:36:23.831294 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30b809-ed52-48df-a0b4-35587b4f8f3f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:23.844888 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30b809-ed52-48df-a0b4-35587b4f8f3f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:23.849864 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5265d-6ff2-42f4-856d-dd89e6baa062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:23.850863 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5265d-6ff2-42f4-856d-dd89e6baa062 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:23.851607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5265d-6ff2-42f4-856d-dd89e6baa062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:23.851896 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5265d-6ff2-42f4-856d-dd89e6baa062 admin admin] PATH_INFO: `/v3/projects/c4e78ab65e134da28437061748b63993/users/b851eb9125d24bb58afdde08ba17177c/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:23.866085 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5265d-6ff2-42f4-856d-dd89e6baa062 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4e78ab65e134da28437061748b63993, user_id=b851eb9125d24bb58afdde08ba17177c, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:23.869716 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5265d-6ff2-42f4-856d-dd89e6baa062 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:23.883316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1236/2455] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:36:23 2022] PUT /identity/v3/projects/c4e78ab65e134da28437061748b63993/users/b851eb9125d24bb58afdde08ba17177c/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:23.893375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1a681-7d8f-4c8f-bb29-899993f86eb2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:23.893884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1a681-7d8f-4c8f-bb29-899993f86eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:23.894018 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1a681-7d8f-4c8f-bb29-899993f86eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:23.920622 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-15f1a681-7d8f-4c8f-bb29-899993f86eb2 None None] MFA Rules not processed for user `b851eb9125d24bb58afdde08ba17177c`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:36:23.921853 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-15f1a681-7d8f-4c8f-bb29-899993f86eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:23.986035 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1220/2456] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:36:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:36:23.990483 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5265d-6ff2-42f4-856d-dd89e6baa062 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:24.057375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5265d-6ff2-42f4-856d-dd89e6baa062 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:24.061968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf0463-4a22-461e-9a08-cae2ee594de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:24.062618 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf0463-4a22-461e-9a08-cae2ee594de0 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:24.062906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf0463-4a22-461e-9a08-cae2ee594de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:24.063181 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf0463-4a22-461e-9a08-cae2ee594de0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:24.063868 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf0463-4a22-461e-9a08-cae2ee594de0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:24.065824 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf0463-4a22-461e-9a08-cae2ee594de0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:24.084530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:24.096190 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1237/2457] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:36:23 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:36:24.097381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1a681-7d8f-4c8f-bb29-899993f86eb2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:24.108197 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1a681-7d8f-4c8f-bb29-899993f86eb2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:24.154761 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:24.171325 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f82b9-4347-475e-90f0-f205c0aaa224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:24.171991 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f82b9-4347-475e-90f0-f205c0aaa224 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:24.172276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f82b9-4347-475e-90f0-f205c0aaa224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:24.172554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f82b9-4347-475e-90f0-f205c0aaa224 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:24.173270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01f82b9-4347-475e-90f0-f205c0aaa224 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:24.175649 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01f82b9-4347-475e-90f0-f205c0aaa224 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:24.197466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1221/2458] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:36:24 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:36:24.201411 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf0463-4a22-461e-9a08-cae2ee594de0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:24.212560 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf0463-4a22-461e-9a08-cae2ee594de0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:24.217142 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4228d-3e5c-452c-aaf1-66a3edc513de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:24.217852 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4228d-3e5c-452c-aaf1-66a3edc513de admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:24.218173 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4228d-3e5c-452c-aaf1-66a3edc513de admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:24.218453 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4228d-3e5c-452c-aaf1-66a3edc513de admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:24.219177 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd4228d-3e5c-452c-aaf1-66a3edc513de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:24.220777 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd4228d-3e5c-452c-aaf1-66a3edc513de admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:24.231435 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1238/2459] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:36:24.235434 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f82b9-4347-475e-90f0-f205c0aaa224 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:24.245624 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f82b9-4347-475e-90f0-f205c0aaa224 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:24.250299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bafa6e-6dc1-42d7-a64d-05adbb4f7273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:24.251161 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bafa6e-6dc1-42d7-a64d-05adbb4f7273 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:24.251449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bafa6e-6dc1-42d7-a64d-05adbb4f7273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:24.251758 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bafa6e-6dc1-42d7-a64d-05adbb4f7273 admin admin] PATH_INFO: `/v3/projects/18eb1f5911254cdf9e197fb4083df4b7/users/bda1c4a80a034a79aba4dffd105e1c9f/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:24.267400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bafa6e-6dc1-42d7-a64d-05adbb4f7273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18eb1f5911254cdf9e197fb4083df4b7, user_id=bda1c4a80a034a79aba4dffd105e1c9f, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:24.270392 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bafa6e-6dc1-42d7-a64d-05adbb4f7273 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:24.284425 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1222/2460] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:36:24 2022] PUT /identity/v3/projects/18eb1f5911254cdf9e197fb4083df4b7/users/bda1c4a80a034a79aba4dffd105e1c9f/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:24.288465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4228d-3e5c-452c-aaf1-66a3edc513de admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:24.300766 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4228d-3e5c-452c-aaf1-66a3edc513de admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:24.378269 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b5a8b-9e18-4ccc-8355-27d88a833105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:24.378854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b5a8b-9e18-4ccc-8355-27d88a833105 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:24.379141 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b5a8b-9e18-4ccc-8355-27d88a833105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:24.379420 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b5a8b-9e18-4ccc-8355-27d88a833105 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:24.380137 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b5a8b-9e18-4ccc-8355-27d88a833105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:24.381768 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b5a8b-9e18-4ccc-8355-27d88a833105 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:24.392130 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1239/2461] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:36:24.396365 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bafa6e-6dc1-42d7-a64d-05adbb4f7273 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:24.409264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bafa6e-6dc1-42d7-a64d-05adbb4f7273 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:24.415933 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b28ca-307f-49a2-b028-5bd68e6df02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:24.416829 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b28ca-307f-49a2-b028-5bd68e6df02b admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:24.417116 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b28ca-307f-49a2-b028-5bd68e6df02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:24.417398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b28ca-307f-49a2-b028-5bd68e6df02b admin admin] PATH_INFO: `/v3/projects/18eb1f5911254cdf9e197fb4083df4b7/users/bda1c4a80a034a79aba4dffd105e1c9f/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:24.419454 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b28ca-307f-49a2-b028-5bd68e6df02b admin admin] RBAC: Authorizing `identity:create_grant(project_id=18eb1f5911254cdf9e197fb4083df4b7, user_id=bda1c4a80a034a79aba4dffd105e1c9f, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:24.422170 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b28ca-307f-49a2-b028-5bd68e6df02b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:24.434755 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1223/2462] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:36:24 2022] PUT /identity/v3/projects/18eb1f5911254cdf9e197fb4083df4b7/users/bda1c4a80a034a79aba4dffd105e1c9f/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:24.446524 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3934b-b019-4932-980e-13fcfb2b99d7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:24.446809 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3934b-b019-4932-980e-13fcfb2b99d7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:24.447091 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3934b-b019-4932-980e-13fcfb2b99d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:24.476145 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fbf3934b-b019-4932-980e-13fcfb2b99d7 None None] MFA Rules not processed for user `bda1c4a80a034a79aba4dffd105e1c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:36:24.477382 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fbf3934b-b019-4932-980e-13fcfb2b99d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:24.559323 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1240/2463] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:36:24 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:36:24.572421 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b28ca-307f-49a2-b028-5bd68e6df02b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:24.574237 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f25b28ca-307f-49a2-b028-5bd68e6df02b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:24.651367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b28ca-307f-49a2-b028-5bd68e6df02b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:24.657266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ba239-46a4-4281-aeee-ae71dd3649e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:24.659187 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ba239-46a4-4281-aeee-ae71dd3649e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:24.659492 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ba239-46a4-4281-aeee-ae71dd3649e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:24.660053 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ba239-46a4-4281-aeee-ae71dd3649e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:24.673168 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ba239-46a4-4281-aeee-ae71dd3649e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:24.674664 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ba239-46a4-4281-aeee-ae71dd3649e0 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:24.686576 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1224/2464] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:36:24 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:36:24.714979 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3934b-b019-4932-980e-13fcfb2b99d7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:24.780941 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3934b-b019-4932-980e-13fcfb2b99d7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:24.785649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7051f-824d-42e8-aa0f-7ecdf0573060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:24.786550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7051f-824d-42e8-aa0f-7ecdf0573060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:24.786841 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7051f-824d-42e8-aa0f-7ecdf0573060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:24.787125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7051f-824d-42e8-aa0f-7ecdf0573060 admin admin] PATH_INFO: `/v3/users/b851eb9125d24bb58afdde08ba17177c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:24.788860 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7051f-824d-42e8-aa0f-7ecdf0573060 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b851eb9125d24bb58afdde08ba17177c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:24.790978 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7051f-824d-42e8-aa0f-7ecdf0573060 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:24.804372 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:24.841645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-95b7051f-824d-42e8-aa0f-7ecdf0573060 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b851eb9125d24bb58afdde08ba17177c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:24.854247 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-95b7051f-824d-42e8-aa0f-7ecdf0573060 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b851eb9125d24bb58afdde08ba17177c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:24.854965 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-95b7051f-824d-42e8-aa0f-7ecdf0573060 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b851eb9125d24bb58afdde08ba17177c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:24.861810 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-95b7051f-824d-42e8-aa0f-7ecdf0573060 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b851eb9125d24bb58afdde08ba17177c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:24.862459 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-95b7051f-824d-42e8-aa0f-7ecdf0573060 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b851eb9125d24bb58afdde08ba17177c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:24.878201 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1241/2465] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:36:24 2022] DELETE /identity/v3/users/b851eb9125d24bb58afdde08ba17177c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:24.882888 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a59ea-712f-4c97-8d2f-06190e6131b3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:24.883286 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a59ea-712f-4c97-8d2f-06190e6131b3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:24.883563 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a59ea-712f-4c97-8d2f-06190e6131b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:24.903910 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8b9a59ea-712f-4c97-8d2f-06190e6131b3 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:36:24.905038 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8b9a59ea-712f-4c97-8d2f-06190e6131b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:24.968207 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1225/2466] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:36:24.978713 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7051f-824d-42e8-aa0f-7ecdf0573060 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:24.980132 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-95b7051f-824d-42e8-aa0f-7ecdf0573060 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:25.041299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7051f-824d-42e8-aa0f-7ecdf0573060 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:25.046556 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8394bca-98b7-47ef-be4c-537559bba780 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:25.047205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8394bca-98b7-47ef-be4c-537559bba780 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:25.047518 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8394bca-98b7-47ef-be4c-537559bba780 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:25.047802 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8394bca-98b7-47ef-be4c-537559bba780 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:25.057923 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8394bca-98b7-47ef-be4c-537559bba780 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:25.059661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8394bca-98b7-47ef-be4c-537559bba780 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:25.070955 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1242/2467] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:36:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:36:25.361476 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a59ea-712f-4c97-8d2f-06190e6131b3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:25.362971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8b9a59ea-712f-4c97-8d2f-06190e6131b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:25.378418 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a59ea-712f-4c97-8d2f-06190e6131b3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:25.384406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3a9aa-e75d-4668-9f57-eef0e98357f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:25.385440 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3a9aa-e75d-4668-9f57-eef0e98357f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:25.385768 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3a9aa-e75d-4668-9f57-eef0e98357f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:25.386050 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3a9aa-e75d-4668-9f57-eef0e98357f5 admin admin] PATH_INFO: `/v3/projects/c4e78ab65e134da28437061748b63993` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:25.387278 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3a9aa-e75d-4668-9f57-eef0e98357f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4e78ab65e134da28437061748b63993)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:25.389283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3a9aa-e75d-4668-9f57-eef0e98357f5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:25.443456 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:25.454922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-51d3a9aa-e75d-4668-9f57-eef0e98357f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4e78ab65e134da28437061748b63993', 'request_id': 'req-51d3a9aa-e75d-4668-9f57-eef0e98357f5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:25.462541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-51d3a9aa-e75d-4668-9f57-eef0e98357f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4e78ab65e134da28437061748b63993', 'request_id': 'req-51d3a9aa-e75d-4668-9f57-eef0e98357f5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:25.468943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-51d3a9aa-e75d-4668-9f57-eef0e98357f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4e78ab65e134da28437061748b63993', 'request_id': 'req-51d3a9aa-e75d-4668-9f57-eef0e98357f5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:25.483171 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-51d3a9aa-e75d-4668-9f57-eef0e98357f5 admin admin] The token cache is being invalidate because project c4e78ab65e134da28437061748b63993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:36:25.483464 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-51d3a9aa-e75d-4668-9f57-eef0e98357f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:25.484087 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-51d3a9aa-e75d-4668-9f57-eef0e98357f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:25.485675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1226/2468] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:36:25 2022] DELETE /identity/v3/projects/c4e78ab65e134da28437061748b63993 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:25.489957 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8394bca-98b7-47ef-be4c-537559bba780 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:25.500774 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8394bca-98b7-47ef-be4c-537559bba780 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:25.560722 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30817405-c235-4e45-b918-e3af565768d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:25.561602 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30817405-c235-4e45-b918-e3af565768d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:25.561911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30817405-c235-4e45-b918-e3af565768d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:25.562196 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30817405-c235-4e45-b918-e3af565768d1 admin admin] PATH_INFO: `/v3/users/bda1c4a80a034a79aba4dffd105e1c9f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:25.563384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30817405-c235-4e45-b918-e3af565768d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bda1c4a80a034a79aba4dffd105e1c9f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:25.565318 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30817405-c235-4e45-b918-e3af565768d1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:25.598432 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:25.612304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-30817405-c235-4e45-b918-e3af565768d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bda1c4a80a034a79aba4dffd105e1c9f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:25.624607 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-30817405-c235-4e45-b918-e3af565768d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bda1c4a80a034a79aba4dffd105e1c9f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:25.625314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-30817405-c235-4e45-b918-e3af565768d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bda1c4a80a034a79aba4dffd105e1c9f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:25.632193 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-30817405-c235-4e45-b918-e3af565768d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bda1c4a80a034a79aba4dffd105e1c9f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:25.632879 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-30817405-c235-4e45-b918-e3af565768d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bda1c4a80a034a79aba4dffd105e1c9f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:25.645477 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1243/2469] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:36:25 2022] DELETE /identity/v3/users/bda1c4a80a034a79aba4dffd105e1c9f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:25.922427 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3a9aa-e75d-4668-9f57-eef0e98357f5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:25.923957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-51d3a9aa-e75d-4668-9f57-eef0e98357f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:25.936902 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3a9aa-e75d-4668-9f57-eef0e98357f5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:26.000727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741cd1b-4230-4da5-b276-95b897967844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:26.001491 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741cd1b-4230-4da5-b276-95b897967844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:26.001835 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741cd1b-4230-4da5-b276-95b897967844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:26.002114 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741cd1b-4230-4da5-b276-95b897967844 admin admin] PATH_INFO: `/v3/projects/18eb1f5911254cdf9e197fb4083df4b7` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:26.003320 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741cd1b-4230-4da5-b276-95b897967844 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18eb1f5911254cdf9e197fb4083df4b7)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:26.005241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741cd1b-4230-4da5-b276-95b897967844 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:26.052131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9741cd1b-4230-4da5-b276-95b897967844 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18eb1f5911254cdf9e197fb4083df4b7', 'request_id': 'req-9741cd1b-4230-4da5-b276-95b897967844'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:26.058124 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9741cd1b-4230-4da5-b276-95b897967844 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18eb1f5911254cdf9e197fb4083df4b7', 'request_id': 'req-9741cd1b-4230-4da5-b276-95b897967844'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:26.063638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9741cd1b-4230-4da5-b276-95b897967844 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18eb1f5911254cdf9e197fb4083df4b7', 'request_id': 'req-9741cd1b-4230-4da5-b276-95b897967844'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:26.076061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9741cd1b-4230-4da5-b276-95b897967844 admin admin] The token cache is being invalidate because project 18eb1f5911254cdf9e197fb4083df4b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:36:26.076355 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9741cd1b-4230-4da5-b276-95b897967844 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:26.076964 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9741cd1b-4230-4da5-b276-95b897967844 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:26.078524 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1227/2470] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:36:25 2022] DELETE /identity/v3/projects/18eb1f5911254cdf9e197fb4083df4b7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:26.083517 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0122e-41c1-4565-8a5e-175fa4908fbb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:26.084033 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0122e-41c1-4565-8a5e-175fa4908fbb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:26.084143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0122e-41c1-4565-8a5e-175fa4908fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:26.104109 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-17e0122e-41c1-4565-8a5e-175fa4908fbb None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:36:26.105316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-17e0122e-41c1-4565-8a5e-175fa4908fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:26.164116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1244/2471] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:36:26.171684 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4dd24-b655-4d4a-89d0-406552d7760b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:26.171975 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4dd24-b655-4d4a-89d0-406552d7760b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:26.172257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4dd24-b655-4d4a-89d0-406552d7760b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:26.190387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4fd4dd24-b655-4d4a-89d0-406552d7760b None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:36:26.191593 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4fd4dd24-b655-4d4a-89d0-406552d7760b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:26.201235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1228/2472] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:36:26.205492 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0122e-41c1-4565-8a5e-175fa4908fbb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:26.215993 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0122e-41c1-4565-8a5e-175fa4908fbb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:26.220545 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3512bb7-b392-49e1-86be-0f7dac5e2d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:26.221187 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3512bb7-b392-49e1-86be-0f7dac5e2d67 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:26.221475 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3512bb7-b392-49e1-86be-0f7dac5e2d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:26.221789 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3512bb7-b392-49e1-86be-0f7dac5e2d67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:26.222512 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3512bb7-b392-49e1-86be-0f7dac5e2d67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:36:26.222819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3512bb7-b392-49e1-86be-0f7dac5e2d67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:26.224427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3512bb7-b392-49e1-86be-0f7dac5e2d67 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:26.239198 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1245/2473] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:36:26.240682 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4dd24-b655-4d4a-89d0-406552d7760b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:26.249997 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4dd24-b655-4d4a-89d0-406552d7760b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:26.254703 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4da2a1-e859-4828-a193-d331b2a9031f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:26.255290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4da2a1-e859-4828-a193-d331b2a9031f admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:26.255575 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4da2a1-e859-4828-a193-d331b2a9031f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:26.255855 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4da2a1-e859-4828-a193-d331b2a9031f admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:26.256539 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4da2a1-e859-4828-a193-d331b2a9031f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:26.259102 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4da2a1-e859-4828-a193-d331b2a9031f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:26.286111 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1229/2474] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:36:26 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:36:26.290183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3512bb7-b392-49e1-86be-0f7dac5e2d67 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:26.300118 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3512bb7-b392-49e1-86be-0f7dac5e2d67 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:26.358326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a9616-943e-4fe3-8036-5ffa8d27caef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:26.358985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a9616-943e-4fe3-8036-5ffa8d27caef admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:26.359269 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a9616-943e-4fe3-8036-5ffa8d27caef admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:26.359550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a9616-943e-4fe3-8036-5ffa8d27caef admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:26.360233 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a9616-943e-4fe3-8036-5ffa8d27caef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:26.362225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a9616-943e-4fe3-8036-5ffa8d27caef admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:26.383666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1246/2475] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:36:26 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:36:26.386149 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4da2a1-e859-4828-a193-d331b2a9031f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:26.396353 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4da2a1-e859-4828-a193-d331b2a9031f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:26.401747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694f06f-9e08-4b40-a05f-f27707f8409f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:26.402390 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694f06f-9e08-4b40-a05f-f27707f8409f admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:26.402670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694f06f-9e08-4b40-a05f-f27707f8409f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:26.402943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694f06f-9e08-4b40-a05f-f27707f8409f admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:26.403659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d694f06f-9e08-4b40-a05f-f27707f8409f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:26.405250 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d694f06f-9e08-4b40-a05f-f27707f8409f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:26.415443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1230/2476] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:36:26.419270 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a9616-943e-4fe3-8036-5ffa8d27caef admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:26.429322 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a9616-943e-4fe3-8036-5ffa8d27caef admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:26.434024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078b8a3-b5c0-48d5-8b6d-5eb9512c2e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:26.434674 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078b8a3-b5c0-48d5-8b6d-5eb9512c2e8a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:26.434964 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078b8a3-b5c0-48d5-8b6d-5eb9512c2e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:26.435248 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078b8a3-b5c0-48d5-8b6d-5eb9512c2e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:26.435973 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0078b8a3-b5c0-48d5-8b6d-5eb9512c2e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:26.437618 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0078b8a3-b5c0-48d5-8b6d-5eb9512c2e8a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:26.448780 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1247/2477] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:36:26.452982 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694f06f-9e08-4b40-a05f-f27707f8409f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:26.463711 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694f06f-9e08-4b40-a05f-f27707f8409f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:26.468382 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fad97-eaf8-436a-a668-4c4a927189a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:26.469247 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fad97-eaf8-436a-a668-4c4a927189a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:26.469533 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fad97-eaf8-436a-a668-4c4a927189a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:26.469850 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fad97-eaf8-436a-a668-4c4a927189a6 admin admin] PATH_INFO: `/v3/projects/661a902391de41d0befafdb2ed53a9a0/users/790bd55981a14f6c94ea69a83e4a5822/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:26.480876 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:26.485383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008fad97-eaf8-436a-a668-4c4a927189a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=661a902391de41d0befafdb2ed53a9a0, user_id=790bd55981a14f6c94ea69a83e4a5822, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:26.488446 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008fad97-eaf8-436a-a668-4c4a927189a6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:26.502402 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1231/2478] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:36:26 2022] PUT /identity/v3/projects/661a902391de41d0befafdb2ed53a9a0/users/790bd55981a14f6c94ea69a83e4a5822/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:26.512881 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c7a3f-3585-4a06-923f-3226c1f9659c None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:26.513121 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c7a3f-3585-4a06-923f-3226c1f9659c None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:26.513431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c7a3f-3585-4a06-923f-3226c1f9659c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:26.541672 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-855c7a3f-3585-4a06-923f-3226c1f9659c None None] MFA Rules not processed for user `790bd55981a14f6c94ea69a83e4a5822`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:36:26.542937 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-855c7a3f-3585-4a06-923f-3226c1f9659c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:26.546947 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:26.609544 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1248/2479] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:36:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:36:26.614028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fad97-eaf8-436a-a668-4c4a927189a6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:26.678299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fad97-eaf8-436a-a668-4c4a927189a6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:26.683499 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957592df-9b8b-44ac-8330-f794e4ea48c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:26.684148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957592df-9b8b-44ac-8330-f794e4ea48c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:26.684434 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957592df-9b8b-44ac-8330-f794e4ea48c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:26.684716 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957592df-9b8b-44ac-8330-f794e4ea48c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:26.685401 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957592df-9b8b-44ac-8330-f794e4ea48c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:26.687388 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957592df-9b8b-44ac-8330-f794e4ea48c8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:26.713133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1232/2480] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:36:26 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:36:26.717254 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c7a3f-3585-4a06-923f-3226c1f9659c None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:26.728573 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c7a3f-3585-4a06-923f-3226c1f9659c None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:26.783566 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35dca9-aa75-4386-bd4c-298bc908a665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:26.784288 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35dca9-aa75-4386-bd4c-298bc908a665 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:26.784599 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35dca9-aa75-4386-bd4c-298bc908a665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:26.784878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35dca9-aa75-4386-bd4c-298bc908a665 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:26.785614 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f35dca9-aa75-4386-bd4c-298bc908a665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:26.787535 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f35dca9-aa75-4386-bd4c-298bc908a665 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:26.807470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1249/2481] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:36:26 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:36:26.811425 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957592df-9b8b-44ac-8330-f794e4ea48c8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:26.822208 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957592df-9b8b-44ac-8330-f794e4ea48c8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:26.826800 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcae445-995e-48ea-b294-5330875f0b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:26.827442 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcae445-995e-48ea-b294-5330875f0b90 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:26.827730 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcae445-995e-48ea-b294-5330875f0b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:26.828006 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcae445-995e-48ea-b294-5330875f0b90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:26.828724 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcae445-995e-48ea-b294-5330875f0b90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:26.830356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcae445-995e-48ea-b294-5330875f0b90 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:26.841116 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1233/2482] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:36:26.844788 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35dca9-aa75-4386-bd4c-298bc908a665 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:26.854874 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35dca9-aa75-4386-bd4c-298bc908a665 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:26.859385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80501806-d351-4d12-91e8-ff5354d16ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:26.860026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80501806-d351-4d12-91e8-ff5354d16ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:26.860310 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80501806-d351-4d12-91e8-ff5354d16ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:26.860590 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80501806-d351-4d12-91e8-ff5354d16ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:26.861304 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80501806-d351-4d12-91e8-ff5354d16ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:26.862927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80501806-d351-4d12-91e8-ff5354d16ba8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:26.873208 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1250/2483] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:36:26.877174 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcae445-995e-48ea-b294-5330875f0b90 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:26.887829 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcae445-995e-48ea-b294-5330875f0b90 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:26.892385 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a799b-ef25-436c-af2a-92ca58cd3ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:26.893238 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a799b-ef25-436c-af2a-92ca58cd3ce9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:26.893524 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a799b-ef25-436c-af2a-92ca58cd3ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:26.893834 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a799b-ef25-436c-af2a-92ca58cd3ce9 admin admin] PATH_INFO: `/v3/projects/88ef28246da14d09878278d7d5facb63/users/04d773b872044d64894cf44d54a796fc/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:26.907795 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a799b-ef25-436c-af2a-92ca58cd3ce9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88ef28246da14d09878278d7d5facb63, user_id=04d773b872044d64894cf44d54a796fc, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:26.910492 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a799b-ef25-436c-af2a-92ca58cd3ce9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:26.923842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1234/2484] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:36:26 2022] PUT /identity/v3/projects/88ef28246da14d09878278d7d5facb63/users/04d773b872044d64894cf44d54a796fc/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:26.936288 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f136dd1-2366-4429-ab43-0ae041501e64 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:26.936576 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f136dd1-2366-4429-ab43-0ae041501e64 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:26.936878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f136dd1-2366-4429-ab43-0ae041501e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:26.962772 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7f136dd1-2366-4429-ab43-0ae041501e64 None None] MFA Rules not processed for user `04d773b872044d64894cf44d54a796fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:36:26.963978 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7f136dd1-2366-4429-ab43-0ae041501e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:27.028367 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1251/2485] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:36:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:36:27.032860 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a799b-ef25-436c-af2a-92ca58cd3ce9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:27.094062 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a799b-ef25-436c-af2a-92ca58cd3ce9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:27.098653 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4ddf5-bb74-4b34-81b7-5015c264ea89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:27.099300 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4ddf5-bb74-4b34-81b7-5015c264ea89 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:27.099589 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4ddf5-bb74-4b34-81b7-5015c264ea89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:27.099869 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4ddf5-bb74-4b34-81b7-5015c264ea89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:27.100549 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba4ddf5-bb74-4b34-81b7-5015c264ea89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:27.103071 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba4ddf5-bb74-4b34-81b7-5015c264ea89 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:27.128498 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1235/2486] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:36:27 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:36:27.133037 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f136dd1-2366-4429-ab43-0ae041501e64 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:27.143972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f136dd1-2366-4429-ab43-0ae041501e64 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:27.152853 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:27.202314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac190bcb-3c13-41ed-bc4e-28efc20d08ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:27.202980 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac190bcb-3c13-41ed-bc4e-28efc20d08ab admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:27.203265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac190bcb-3c13-41ed-bc4e-28efc20d08ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:27.203542 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac190bcb-3c13-41ed-bc4e-28efc20d08ab admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:27.204243 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac190bcb-3c13-41ed-bc4e-28efc20d08ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:27.206236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac190bcb-3c13-41ed-bc4e-28efc20d08ab admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:27.229461 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1252/2487] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:36:27 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:36:27.233572 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4ddf5-bb74-4b34-81b7-5015c264ea89 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:27.244333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4ddf5-bb74-4b34-81b7-5015c264ea89 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:27.248894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5e82d-383a-406f-bdf1-e21ed1e984ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:27.249539 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5e82d-383a-406f-bdf1-e21ed1e984ca admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:27.249849 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5e82d-383a-406f-bdf1-e21ed1e984ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:27.250126 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5e82d-383a-406f-bdf1-e21ed1e984ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:27.250844 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5e82d-383a-406f-bdf1-e21ed1e984ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:27.252436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5e82d-383a-406f-bdf1-e21ed1e984ca admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:27.262754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1236/2488] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:36:27.266844 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac190bcb-3c13-41ed-bc4e-28efc20d08ab admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:27.277706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac190bcb-3c13-41ed-bc4e-28efc20d08ab admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:27.282398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41246e-dceb-4947-9f93-8db2ee632bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:27.283274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41246e-dceb-4947-9f93-8db2ee632bcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:27.283560 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41246e-dceb-4947-9f93-8db2ee632bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:27.283843 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41246e-dceb-4947-9f93-8db2ee632bcc admin admin] PATH_INFO: `/v3/projects/770c3f6e9e924fafaa7713b654e55757/users/c4429e25138449a3b02f60da5f1c2e8d/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:27.298875 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41246e-dceb-4947-9f93-8db2ee632bcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=770c3f6e9e924fafaa7713b654e55757, user_id=c4429e25138449a3b02f60da5f1c2e8d, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:27.301550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41246e-dceb-4947-9f93-8db2ee632bcc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:27.314526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1253/2489] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:36:27 2022] PUT /identity/v3/projects/770c3f6e9e924fafaa7713b654e55757/users/c4429e25138449a3b02f60da5f1c2e8d/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:36:27.318434 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5e82d-383a-406f-bdf1-e21ed1e984ca admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:27.328537 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5e82d-383a-406f-bdf1-e21ed1e984ca admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:27.337985 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:27.386664 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d8f929-84f4-4ef0-9ea3-6b3441f27327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:27.387250 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8f929-84f4-4ef0-9ea3-6b3441f27327 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:27.387533 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8f929-84f4-4ef0-9ea3-6b3441f27327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:27.387818 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8f929-84f4-4ef0-9ea3-6b3441f27327 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:27.388557 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d8f929-84f4-4ef0-9ea3-6b3441f27327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:27.390188 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d8f929-84f4-4ef0-9ea3-6b3441f27327 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:27.400292 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1237/2490] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:36:27.404487 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41246e-dceb-4947-9f93-8db2ee632bcc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:27.416012 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41246e-dceb-4947-9f93-8db2ee632bcc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:27.420648 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3ef18-165f-4abf-a3c2-e8c89b9ebc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:27.421552 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3ef18-165f-4abf-a3c2-e8c89b9ebc1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:27.421873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3ef18-165f-4abf-a3c2-e8c89b9ebc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:27.422154 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3ef18-165f-4abf-a3c2-e8c89b9ebc1d admin admin] PATH_INFO: `/v3/projects/770c3f6e9e924fafaa7713b654e55757/users/c4429e25138449a3b02f60da5f1c2e8d/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:27.424270 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3ef18-165f-4abf-a3c2-e8c89b9ebc1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=770c3f6e9e924fafaa7713b654e55757, user_id=c4429e25138449a3b02f60da5f1c2e8d, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:27.427007 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3ef18-165f-4abf-a3c2-e8c89b9ebc1d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:27.437505 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1254/2491] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:36:27 2022] PUT /identity/v3/projects/770c3f6e9e924fafaa7713b654e55757/users/c4429e25138449a3b02f60da5f1c2e8d/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:27.447865 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1dc8a-6031-488c-a8d8-62184c542a94 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:27.448158 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1dc8a-6031-488c-a8d8-62184c542a94 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:27.448441 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1dc8a-6031-488c-a8d8-62184c542a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:27.475180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-caa1dc8a-6031-488c-a8d8-62184c542a94 None None] MFA Rules not processed for user `c4429e25138449a3b02f60da5f1c2e8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:36:27.477517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-caa1dc8a-6031-488c-a8d8-62184c542a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:27.548792 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1238/2492] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:36:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:36:27.560059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3ef18-165f-4abf-a3c2-e8c89b9ebc1d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:27.561791 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-02e3ef18-165f-4abf-a3c2-e8c89b9ebc1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:27.624556 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3ef18-165f-4abf-a3c2-e8c89b9ebc1d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:27.628406 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3fa33a-7360-45b3-9912-c9eae01bc865 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:27.629258 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fa33a-7360-45b3-9912-c9eae01bc865 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:27.629401 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fa33a-7360-45b3-9912-c9eae01bc865 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:27.629668 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fa33a-7360-45b3-9912-c9eae01bc865 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:27.640079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3fa33a-7360-45b3-9912-c9eae01bc865 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:27.641448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3fa33a-7360-45b3-9912-c9eae01bc865 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:27.651915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1255/2493] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:36:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:36:27.690525 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1dc8a-6031-488c-a8d8-62184c542a94 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:27.691955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-caa1dc8a-6031-488c-a8d8-62184c542a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:27.704796 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1dc8a-6031-488c-a8d8-62184c542a94 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:27.708416 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f83fa-d2b9-4237-b0a7-5ea193758f1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:27.709047 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f83fa-d2b9-4237-b0a7-5ea193758f1a service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:27.709329 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f83fa-d2b9-4237-b0a7-5ea193758f1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:27.709630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f83fa-d2b9-4237-b0a7-5ea193758f1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:27.768330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f83fa-d2b9-4237-b0a7-5ea193758f1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:27.769826 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f83fa-d2b9-4237-b0a7-5ea193758f1a service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:27.779356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1239/2494] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:36:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:36:30.623781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3fa33a-7360-45b3-9912-c9eae01bc865 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:30.684447 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3fa33a-7360-45b3-9912-c9eae01bc865 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:30.689179 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bd783-073e-4267-9cc3-138ae971d32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:30.690075 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bd783-073e-4267-9cc3-138ae971d32f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:30.690355 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bd783-073e-4267-9cc3-138ae971d32f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:30.690622 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bd783-073e-4267-9cc3-138ae971d32f admin admin] PATH_INFO: `/v3/users/790bd55981a14f6c94ea69a83e4a5822` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:30.691842 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6bd783-073e-4267-9cc3-138ae971d32f admin admin] RBAC: Authorizing `identity:delete_user(user_id=790bd55981a14f6c94ea69a83e4a5822)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:30.693800 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6bd783-073e-4267-9cc3-138ae971d32f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:30.704825 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:30.741205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bd6bd783-073e-4267-9cc3-138ae971d32f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '790bd55981a14f6c94ea69a83e4a5822'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:30.752906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bd6bd783-073e-4267-9cc3-138ae971d32f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '790bd55981a14f6c94ea69a83e4a5822'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:30.753608 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bd6bd783-073e-4267-9cc3-138ae971d32f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '790bd55981a14f6c94ea69a83e4a5822'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:30.760990 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bd6bd783-073e-4267-9cc3-138ae971d32f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '790bd55981a14f6c94ea69a83e4a5822'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:30.761659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bd6bd783-073e-4267-9cc3-138ae971d32f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '790bd55981a14f6c94ea69a83e4a5822'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:30.776216 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1256/2495] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:36:30 2022] DELETE /identity/v3/users/790bd55981a14f6c94ea69a83e4a5822 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:30.780908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908312b-3de8-4077-871e-bca7eb47d9b9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:30.781419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908312b-3de8-4077-871e-bca7eb47d9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:30.781536 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908312b-3de8-4077-871e-bca7eb47d9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:30.798241 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6908312b-3de8-4077-871e-bca7eb47d9b9 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:36:30.799476 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6908312b-3de8-4077-871e-bca7eb47d9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:30.827924 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:30.858797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1240/2496] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:36:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:36:30.868535 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bd783-073e-4267-9cc3-138ae971d32f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:30.869966 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bd6bd783-073e-4267-9cc3-138ae971d32f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:30.926874 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bd783-073e-4267-9cc3-138ae971d32f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:30.931431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7fbf2-5eef-44db-bde1-4ebe601839b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:30.932085 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7fbf2-5eef-44db-bde1-4ebe601839b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:30.932395 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7fbf2-5eef-44db-bde1-4ebe601839b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:30.932683 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7fbf2-5eef-44db-bde1-4ebe601839b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:30.941721 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7fbf2-5eef-44db-bde1-4ebe601839b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:30.943161 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7fbf2-5eef-44db-bde1-4ebe601839b2 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:30.953015 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1257/2497] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:36:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:36:31.233366 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908312b-3de8-4077-871e-bca7eb47d9b9 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:31.234812 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6908312b-3de8-4077-871e-bca7eb47d9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:31.247021 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908312b-3de8-4077-871e-bca7eb47d9b9 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:31.251638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d1233-1719-456a-8807-56d4a0631a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:31.252483 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d1233-1719-456a-8807-56d4a0631a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:31.252774 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d1233-1719-456a-8807-56d4a0631a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:31.253042 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d1233-1719-456a-8807-56d4a0631a31 admin admin] PATH_INFO: `/v3/projects/661a902391de41d0befafdb2ed53a9a0` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:31.254264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d1233-1719-456a-8807-56d4a0631a31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=661a902391de41d0befafdb2ed53a9a0)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:31.256170 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d1233-1719-456a-8807-56d4a0631a31 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:31.309339 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c99d1233-1719-456a-8807-56d4a0631a31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '661a902391de41d0befafdb2ed53a9a0', 'request_id': 'req-c99d1233-1719-456a-8807-56d4a0631a31'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:31.315322 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c99d1233-1719-456a-8807-56d4a0631a31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '661a902391de41d0befafdb2ed53a9a0', 'request_id': 'req-c99d1233-1719-456a-8807-56d4a0631a31'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:31.320876 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c99d1233-1719-456a-8807-56d4a0631a31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '661a902391de41d0befafdb2ed53a9a0', 'request_id': 'req-c99d1233-1719-456a-8807-56d4a0631a31'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:31.333123 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c99d1233-1719-456a-8807-56d4a0631a31 admin admin] The token cache is being invalidate because project 661a902391de41d0befafdb2ed53a9a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:36:31.333413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c99d1233-1719-456a-8807-56d4a0631a31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:31.334178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c99d1233-1719-456a-8807-56d4a0631a31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:31.336049 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1241/2498] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:36:31 2022] DELETE /identity/v3/projects/661a902391de41d0befafdb2ed53a9a0 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:31.340530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7fbf2-5eef-44db-bde1-4ebe601839b2 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:31.350141 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7fbf2-5eef-44db-bde1-4ebe601839b2 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:31.408580 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d4f1e-8136-413f-8b74-9235d3326d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:31.409451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d4f1e-8136-413f-8b74-9235d3326d79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:31.409774 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d4f1e-8136-413f-8b74-9235d3326d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:31.410063 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d4f1e-8136-413f-8b74-9235d3326d79 admin admin] PATH_INFO: `/v3/users/04d773b872044d64894cf44d54a796fc` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:31.411252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d4f1e-8136-413f-8b74-9235d3326d79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04d773b872044d64894cf44d54a796fc)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:31.413196 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d4f1e-8136-413f-8b74-9235d3326d79 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:31.448307 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:31.462475 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6a6d4f1e-8136-413f-8b74-9235d3326d79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04d773b872044d64894cf44d54a796fc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:31.474109 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6a6d4f1e-8136-413f-8b74-9235d3326d79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04d773b872044d64894cf44d54a796fc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:31.474774 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6a6d4f1e-8136-413f-8b74-9235d3326d79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04d773b872044d64894cf44d54a796fc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:31.481747 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6a6d4f1e-8136-413f-8b74-9235d3326d79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04d773b872044d64894cf44d54a796fc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:31.482385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6a6d4f1e-8136-413f-8b74-9235d3326d79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04d773b872044d64894cf44d54a796fc'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:31.494879 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1258/2499] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:36:31 2022] DELETE /identity/v3/users/04d773b872044d64894cf44d54a796fc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:36:31.772928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d1233-1719-456a-8807-56d4a0631a31 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:31.775026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c99d1233-1719-456a-8807-56d4a0631a31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:31.788236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d1233-1719-456a-8807-56d4a0631a31 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:31.844808 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b5850-e0b0-4769-85d2-da222561053d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:31.845694 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b5850-e0b0-4769-85d2-da222561053d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:31.845980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b5850-e0b0-4769-85d2-da222561053d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:31.846277 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b5850-e0b0-4769-85d2-da222561053d admin admin] PATH_INFO: `/v3/projects/88ef28246da14d09878278d7d5facb63` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:31.847492 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b5850-e0b0-4769-85d2-da222561053d admin admin] RBAC: Authorizing `identity:delete_project(project_id=88ef28246da14d09878278d7d5facb63)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:31.849368 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b5850-e0b0-4769-85d2-da222561053d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:31.898687 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a19b5850-e0b0-4769-85d2-da222561053d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88ef28246da14d09878278d7d5facb63', 'request_id': 'req-a19b5850-e0b0-4769-85d2-da222561053d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:31.904863 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a19b5850-e0b0-4769-85d2-da222561053d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88ef28246da14d09878278d7d5facb63', 'request_id': 'req-a19b5850-e0b0-4769-85d2-da222561053d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:31.908692 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:31.911221 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a19b5850-e0b0-4769-85d2-da222561053d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88ef28246da14d09878278d7d5facb63', 'request_id': 'req-a19b5850-e0b0-4769-85d2-da222561053d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:31.925395 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a19b5850-e0b0-4769-85d2-da222561053d admin admin] The token cache is being invalidate because project 88ef28246da14d09878278d7d5facb63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:36:31.925742 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a19b5850-e0b0-4769-85d2-da222561053d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:31.926365 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a19b5850-e0b0-4769-85d2-da222561053d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:31.927932 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1242/2500] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:36:31 2022] DELETE /identity/v3/projects/88ef28246da14d09878278d7d5facb63 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:31.933003 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d4f1e-8136-413f-8b74-9235d3326d79 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:31.934505 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6a6d4f1e-8136-413f-8b74-9235d3326d79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:31.948141 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d4f1e-8136-413f-8b74-9235d3326d79 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:32.003534 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a6879-4d6e-46f5-8a51-f0c3cd1de43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:32.004395 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a6879-4d6e-46f5-8a51-f0c3cd1de43f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:32.004684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a6879-4d6e-46f5-8a51-f0c3cd1de43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:32.004967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a6879-4d6e-46f5-8a51-f0c3cd1de43f admin admin] PATH_INFO: `/v3/users/c4429e25138449a3b02f60da5f1c2e8d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:32.006178 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463a6879-4d6e-46f5-8a51-f0c3cd1de43f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4429e25138449a3b02f60da5f1c2e8d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:32.008115 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463a6879-4d6e-46f5-8a51-f0c3cd1de43f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:32.052807 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-463a6879-4d6e-46f5-8a51-f0c3cd1de43f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4429e25138449a3b02f60da5f1c2e8d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:32.065286 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-463a6879-4d6e-46f5-8a51-f0c3cd1de43f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4429e25138449a3b02f60da5f1c2e8d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:32.065983 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-463a6879-4d6e-46f5-8a51-f0c3cd1de43f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4429e25138449a3b02f60da5f1c2e8d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:32.073522 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-463a6879-4d6e-46f5-8a51-f0c3cd1de43f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4429e25138449a3b02f60da5f1c2e8d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:32.074176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-463a6879-4d6e-46f5-8a51-f0c3cd1de43f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4429e25138449a3b02f60da5f1c2e8d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:32.087745 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1259/2501] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:36:31 2022] DELETE /identity/v3/users/c4429e25138449a3b02f60da5f1c2e8d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:32.366670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b5850-e0b0-4769-85d2-da222561053d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:32.368318 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a19b5850-e0b0-4769-85d2-da222561053d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:32.382573 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b5850-e0b0-4769-85d2-da222561053d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:32.439927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c6b79-d7c1-439c-9a9c-7145148df6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:32.440780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c6b79-d7c1-439c-9a9c-7145148df6fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:32.441075 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c6b79-d7c1-439c-9a9c-7145148df6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:32.441358 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c6b79-d7c1-439c-9a9c-7145148df6fa admin admin] PATH_INFO: `/v3/projects/770c3f6e9e924fafaa7713b654e55757` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:32.442570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c6b79-d7c1-439c-9a9c-7145148df6fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=770c3f6e9e924fafaa7713b654e55757)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:32.444490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c6b79-d7c1-439c-9a9c-7145148df6fa admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:32.490470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dc1c6b79-d7c1-439c-9a9c-7145148df6fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '770c3f6e9e924fafaa7713b654e55757', 'request_id': 'req-dc1c6b79-d7c1-439c-9a9c-7145148df6fa'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:32.496976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dc1c6b79-d7c1-439c-9a9c-7145148df6fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '770c3f6e9e924fafaa7713b654e55757', 'request_id': 'req-dc1c6b79-d7c1-439c-9a9c-7145148df6fa'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:32.502486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dc1c6b79-d7c1-439c-9a9c-7145148df6fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '770c3f6e9e924fafaa7713b654e55757', 'request_id': 'req-dc1c6b79-d7c1-439c-9a9c-7145148df6fa'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:32.515082 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dc1c6b79-d7c1-439c-9a9c-7145148df6fa admin admin] The token cache is being invalidate because project 770c3f6e9e924fafaa7713b654e55757 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:36:32.515362 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dc1c6b79-d7c1-439c-9a9c-7145148df6fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:32.515975 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-dc1c6b79-d7c1-439c-9a9c-7145148df6fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:36:32.517787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1243/2502] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:36:32 2022] DELETE /identity/v3/projects/770c3f6e9e924fafaa7713b654e55757 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:32.524774 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f8ca7-ebb3-4b23-b2dc-4f700aea2843 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:32.524774 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f8ca7-ebb3-4b23-b2dc-4f700aea2843 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:32.525025 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f8ca7-ebb3-4b23-b2dc-4f700aea2843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:32.543437 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1f1f8ca7-ebb3-4b23-b2dc-4f700aea2843 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:36:32.544655 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1f1f8ca7-ebb3-4b23-b2dc-4f700aea2843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:32.570168 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:32.607837 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1260/2503] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:36:32.616237 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213dc44d-f633-4207-9eed-946df9bb4555 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:32.616482 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213dc44d-f633-4207-9eed-946df9bb4555 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:32.616885 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213dc44d-f633-4207-9eed-946df9bb4555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:32.636401 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-213dc44d-f633-4207-9eed-946df9bb4555 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:36:32.637672 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-213dc44d-f633-4207-9eed-946df9bb4555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:32.645889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1244/2504] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:36:32.650037 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f8ca7-ebb3-4b23-b2dc-4f700aea2843 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:32.660670 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f8ca7-ebb3-4b23-b2dc-4f700aea2843 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:32.665792 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2f976-ebc1-43bc-bef9-32cbd1ef27ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:32.666451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2f976-ebc1-43bc-bef9-32cbd1ef27ab admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:32.666740 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2f976-ebc1-43bc-bef9-32cbd1ef27ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:32.667021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2f976-ebc1-43bc-bef9-32cbd1ef27ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:32.667751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2f976-ebc1-43bc-bef9-32cbd1ef27ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:36:32.668061 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2f976-ebc1-43bc-bef9-32cbd1ef27ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:32.669699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2f976-ebc1-43bc-bef9-32cbd1ef27ab admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:32.685699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1261/2505] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:36:32.686842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213dc44d-f633-4207-9eed-946df9bb4555 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:32.696924 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213dc44d-f633-4207-9eed-946df9bb4555 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:32.701766 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6862865d-c254-4c55-bdb8-bee7d37ed1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:32.702425 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862865d-c254-4c55-bdb8-bee7d37ed1fa admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:32.702713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862865d-c254-4c55-bdb8-bee7d37ed1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:32.702993 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862865d-c254-4c55-bdb8-bee7d37ed1fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:32.703693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6862865d-c254-4c55-bdb8-bee7d37ed1fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:32.705653 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6862865d-c254-4c55-bdb8-bee7d37ed1fa admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:32.732303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1245/2506] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:36:32 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:36:32.736177 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2f976-ebc1-43bc-bef9-32cbd1ef27ab admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:32.747131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2f976-ebc1-43bc-bef9-32cbd1ef27ab admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:32.806888 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f67f3c-ed4c-4696-8329-e7ca3d185f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:32.807561 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f67f3c-ed4c-4696-8329-e7ca3d185f8c admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:32.807850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f67f3c-ed4c-4696-8329-e7ca3d185f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:32.808131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f67f3c-ed4c-4696-8329-e7ca3d185f8c admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:32.808890 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f67f3c-ed4c-4696-8329-e7ca3d185f8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:32.810888 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f67f3c-ed4c-4696-8329-e7ca3d185f8c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:32.832301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1262/2507] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:36:32 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:36:32.837170 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6862865d-c254-4c55-bdb8-bee7d37ed1fa admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:32.847547 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6862865d-c254-4c55-bdb8-bee7d37ed1fa admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:32.852866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e559bc-6d38-4e1c-a990-75baa31c746e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:32.853621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e559bc-6d38-4e1c-a990-75baa31c746e admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:32.854037 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e559bc-6d38-4e1c-a990-75baa31c746e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:32.854386 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e559bc-6d38-4e1c-a990-75baa31c746e admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:32.855201 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e559bc-6d38-4e1c-a990-75baa31c746e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:32.856890 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e559bc-6d38-4e1c-a990-75baa31c746e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:32.867519 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1246/2508] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:36:32.871493 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f67f3c-ed4c-4696-8329-e7ca3d185f8c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:32.882834 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f67f3c-ed4c-4696-8329-e7ca3d185f8c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:32.889229 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e816f-e02a-46ca-9151-6ca90de363ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:32.889465 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e816f-e02a-46ca-9151-6ca90de363ae admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:32.889596 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e816f-e02a-46ca-9151-6ca90de363ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:32.889885 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e816f-e02a-46ca-9151-6ca90de363ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:32.891141 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e816f-e02a-46ca-9151-6ca90de363ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:32.892469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e816f-e02a-46ca-9151-6ca90de363ae admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:32.905487 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1263/2509] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:36:32.909636 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e559bc-6d38-4e1c-a990-75baa31c746e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:32.921730 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e559bc-6d38-4e1c-a990-75baa31c746e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:32.926729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f0a34-5656-4494-b9a9-105acf74dc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:32.927634 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f0a34-5656-4494-b9a9-105acf74dc84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:32.927976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f0a34-5656-4494-b9a9-105acf74dc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:32.928209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f0a34-5656-4494-b9a9-105acf74dc84 admin admin] PATH_INFO: `/v3/projects/afb8b9ee2d2840028ef4f46f7c2894b5/users/450262c78b0b49a0b0bfc9b51e2d3500/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:32.944949 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f0a34-5656-4494-b9a9-105acf74dc84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afb8b9ee2d2840028ef4f46f7c2894b5, user_id=450262c78b0b49a0b0bfc9b51e2d3500, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:32.948501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f0a34-5656-4494-b9a9-105acf74dc84 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:32.967139 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1247/2510] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:36:32 2022] PUT /identity/v3/projects/afb8b9ee2d2840028ef4f46f7c2894b5/users/450262c78b0b49a0b0bfc9b51e2d3500/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:36:32.980444 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4b713-ed4e-4670-b50c-440a37bba5bc None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:32.981030 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4b713-ed4e-4670-b50c-440a37bba5bc None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:32.981030 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4b713-ed4e-4670-b50c-440a37bba5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:33.010234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-eca4b713-ed4e-4670-b50c-440a37bba5bc None None] MFA Rules not processed for user `450262c78b0b49a0b0bfc9b51e2d3500`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:36:33.011478 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-eca4b713-ed4e-4670-b50c-440a37bba5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:33.085305 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1264/2511] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:36:32 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:36:33.097289 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f0a34-5656-4494-b9a9-105acf74dc84 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:33.099029 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fa6f0a34-5656-4494-b9a9-105acf74dc84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:33.114006 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:33.159810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f0a34-5656-4494-b9a9-105acf74dc84 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:33.163483 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c864073-22ab-44d6-b4b0-db49e1b3abe2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:33.164126 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c864073-22ab-44d6-b4b0-db49e1b3abe2 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:33.164412 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c864073-22ab-44d6-b4b0-db49e1b3abe2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:33.165232 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c864073-22ab-44d6-b4b0-db49e1b3abe2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:33.173907 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c864073-22ab-44d6-b4b0-db49e1b3abe2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:33.175359 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c864073-22ab-44d6-b4b0-db49e1b3abe2 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:33.185333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1248/2512] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:36:33 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:36:43.082311 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:43.180198 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=20 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:36:55.402756 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca4b713-ed4e-4670-b50c-440a37bba5bc None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:55.404222 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-eca4b713-ed4e-4670-b50c-440a37bba5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:55.481349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca4b713-ed4e-4670-b50c-440a37bba5bc None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:55.487551 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4349742-83f1-4669-ac9c-6e8cf5221f53 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49325c1733174e0c98b3475a10c9b5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:55.488197 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4349742-83f1-4669-ac9c-6e8cf5221f53 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:55.488483 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4349742-83f1-4669-ac9c-6e8cf5221f53 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:55.488770 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4349742-83f1-4669-ac9c-6e8cf5221f53 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:55.490588 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c4349742-83f1-4669-ac9c-6e8cf5221f53 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:55.503619 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4349742-83f1-4669-ac9c-6e8cf5221f53 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:55.505295 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4349742-83f1-4669-ac9c-6e8cf5221f53 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:55.515949 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1265/2513] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:36:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:36:59.083236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49266ea8-674e-4a6d-a1e9-187be45e5f9e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:59.083769 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49266ea8-674e-4a6d-a1e9-187be45e5f9e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:59.083868 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49266ea8-674e-4a6d-a1e9-187be45e5f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:59.101932 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-49266ea8-674e-4a6d-a1e9-187be45e5f9e None None] MFA Rules not processed for user `6ac4fc6cab4f47ea910ffdbed0a8e5b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:36:59.103141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-49266ea8-674e-4a6d-a1e9-187be45e5f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:59.159201 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1249/2514] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:36:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:36:59.466843 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4349742-83f1-4669-ac9c-6e8cf5221f53 service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:36:59.468258 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c4349742-83f1-4669-ac9c-6e8cf5221f53 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:59.525116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4349742-83f1-4669-ac9c-6e8cf5221f53 service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:36:59.528705 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc9c21-0ce7-4959-860e-8da7175a25c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:36:59.529344 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc9c21-0ce7-4959-860e-8da7175a25c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:36:59.529661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc9c21-0ce7-4959-860e-8da7175a25c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:36:59.529945 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc9c21-0ce7-4959-860e-8da7175a25c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:36:59.532189 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-61fc9c21-0ce7-4959-860e-8da7175a25c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:36:59.543070 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc9c21-0ce7-4959-860e-8da7175a25c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:36:59.544508 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc9c21-0ce7-4959-860e-8da7175a25c8 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:36:59.553185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1266/2515] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:36:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:37:09.151892 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=25 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:37:09.549676 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=58 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:37:12.868238 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c16074-036f-41a8-8284-23a1448d9713 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:12.868736 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c16074-036f-41a8-8284-23a1448d9713 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:12.868931 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c16074-036f-41a8-8284-23a1448d9713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:12.887844 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-62c16074-036f-41a8-8284-23a1448d9713 None None] MFA Rules not processed for user `31c0594d0be84154a601d625bca699cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:37:12.889062 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-62c16074-036f-41a8-8284-23a1448d9713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:12.946365 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1250/2516] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:37:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:37:16.556787 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc9c21-0ce7-4959-860e-8da7175a25c8 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:16.643887 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc9c21-0ce7-4959-860e-8da7175a25c8 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:16.648508 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6666b5-c207-4034-a33d-19e8fb341977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:16.649487 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6666b5-c207-4034-a33d-19e8fb341977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:16.649676 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6666b5-c207-4034-a33d-19e8fb341977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:16.649959 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6666b5-c207-4034-a33d-19e8fb341977 admin admin] PATH_INFO: `/v3/users/450262c78b0b49a0b0bfc9b51e2d3500` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:16.651152 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6666b5-c207-4034-a33d-19e8fb341977 admin admin] RBAC: Authorizing `identity:delete_user(user_id=450262c78b0b49a0b0bfc9b51e2d3500)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:16.653071 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6666b5-c207-4034-a33d-19e8fb341977 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:16.725023 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bd6666b5-c207-4034-a33d-19e8fb341977 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '450262c78b0b49a0b0bfc9b51e2d3500'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:37:16.743210 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bd6666b5-c207-4034-a33d-19e8fb341977 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '450262c78b0b49a0b0bfc9b51e2d3500'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:37:16.745662 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bd6666b5-c207-4034-a33d-19e8fb341977 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '450262c78b0b49a0b0bfc9b51e2d3500'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:37:16.756144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bd6666b5-c207-4034-a33d-19e8fb341977 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '450262c78b0b49a0b0bfc9b51e2d3500'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:37:16.756895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bd6666b5-c207-4034-a33d-19e8fb341977 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '450262c78b0b49a0b0bfc9b51e2d3500'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:37:16.773265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1267/2517] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:37:16 2022] DELETE /identity/v3/users/450262c78b0b49a0b0bfc9b51e2d3500 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:37:16.777889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da334bc-055d-48dd-9ae4-14cb009c7208 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:16.778335 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da334bc-055d-48dd-9ae4-14cb009c7208 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:16.778439 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da334bc-055d-48dd-9ae4-14cb009c7208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:16.799105 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4da334bc-055d-48dd-9ae4-14cb009c7208 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:37:16.800516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4da334bc-055d-48dd-9ae4-14cb009c7208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:16.887285 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1251/2518] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:37:16.908415 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6666b5-c207-4034-a33d-19e8fb341977 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:16.910227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bd6666b5-c207-4034-a33d-19e8fb341977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:16.982963 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6666b5-c207-4034-a33d-19e8fb341977 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:16.987249 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441ef28-46bc-40b5-bb31-8c40fa0c8783 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:16.987985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441ef28-46bc-40b5-bb31-8c40fa0c8783 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:16.988471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441ef28-46bc-40b5-bb31-8c40fa0c8783 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:16.988827 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441ef28-46bc-40b5-bb31-8c40fa0c8783 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:16.999733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3441ef28-46bc-40b5-bb31-8c40fa0c8783 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:17.001710 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3441ef28-46bc-40b5-bb31-8c40fa0c8783 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:17.013287 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1268/2519] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:37:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:37:17.299554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da334bc-055d-48dd-9ae4-14cb009c7208 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:17.301183 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4da334bc-055d-48dd-9ae4-14cb009c7208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:17.323972 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da334bc-055d-48dd-9ae4-14cb009c7208 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:17.333034 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f700c0-ef62-42ea-bf0a-77562c9fccff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:17.334135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f700c0-ef62-42ea-bf0a-77562c9fccff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:17.334417 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f700c0-ef62-42ea-bf0a-77562c9fccff admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:17.334692 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f700c0-ef62-42ea-bf0a-77562c9fccff admin admin] PATH_INFO: `/v3/projects/afb8b9ee2d2840028ef4f46f7c2894b5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:17.335924 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f700c0-ef62-42ea-bf0a-77562c9fccff admin admin] RBAC: Authorizing `identity:delete_project(project_id=afb8b9ee2d2840028ef4f46f7c2894b5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:17.338315 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f700c0-ef62-42ea-bf0a-77562c9fccff admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:17.385997 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f9f700c0-ef62-42ea-bf0a-77562c9fccff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afb8b9ee2d2840028ef4f46f7c2894b5', 'request_id': 'req-f9f700c0-ef62-42ea-bf0a-77562c9fccff'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:37:17.392698 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f9f700c0-ef62-42ea-bf0a-77562c9fccff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afb8b9ee2d2840028ef4f46f7c2894b5', 'request_id': 'req-f9f700c0-ef62-42ea-bf0a-77562c9fccff'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:37:17.398203 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f9f700c0-ef62-42ea-bf0a-77562c9fccff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afb8b9ee2d2840028ef4f46f7c2894b5', 'request_id': 'req-f9f700c0-ef62-42ea-bf0a-77562c9fccff'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:37:17.411786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f9f700c0-ef62-42ea-bf0a-77562c9fccff admin admin] The token cache is being invalidate because project afb8b9ee2d2840028ef4f46f7c2894b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:37:17.412080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f9f700c0-ef62-42ea-bf0a-77562c9fccff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:37:17.412718 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f9f700c0-ef62-42ea-bf0a-77562c9fccff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:37:17.414333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1252/2520] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:37:17 2022] DELETE /identity/v3/projects/afb8b9ee2d2840028ef4f46f7c2894b5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:37:17.419356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bc9d5-a1b2-4be9-8e2b-0786d6cc133b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:17.419646 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bc9d5-a1b2-4be9-8e2b-0786d6cc133b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:17.419927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bc9d5-a1b2-4be9-8e2b-0786d6cc133b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:17.437504 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8d5bc9d5-a1b2-4be9-8e2b-0786d6cc133b None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:37:17.439274 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8d5bc9d5-a1b2-4be9-8e2b-0786d6cc133b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:17.466035 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:37:17.511443 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1269/2521] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:37:17.519476 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc1085-3be1-411e-a74d-d1ae9fc3849f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:17.519768 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc1085-3be1-411e-a74d-d1ae9fc3849f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:17.520054 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc1085-3be1-411e-a74d-d1ae9fc3849f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:17.538902 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-07bc1085-3be1-411e-a74d-d1ae9fc3849f None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:37:17.540166 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-07bc1085-3be1-411e-a74d-d1ae9fc3849f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:17.548312 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1253/2522] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:37:17.553236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bc9d5-a1b2-4be9-8e2b-0786d6cc133b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:17.562751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bc9d5-a1b2-4be9-8e2b-0786d6cc133b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:17.567331 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2263a-5970-43d7-ba03-f0710f7d642b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:17.567975 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2263a-5970-43d7-ba03-f0710f7d642b admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:17.568258 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2263a-5970-43d7-ba03-f0710f7d642b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:17.568536 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2263a-5970-43d7-ba03-f0710f7d642b admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:17.569257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2263a-5970-43d7-ba03-f0710f7d642b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:37:17.569566 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2263a-5970-43d7-ba03-f0710f7d642b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:17.571181 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2263a-5970-43d7-ba03-f0710f7d642b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:17.586688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1270/2523] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:37:17.587932 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc1085-3be1-411e-a74d-d1ae9fc3849f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:17.598924 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc1085-3be1-411e-a74d-d1ae9fc3849f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:17.604071 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5880689-08fd-4b49-a06b-ab5ecab0d2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:17.604799 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5880689-08fd-4b49-a06b-ab5ecab0d2d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:17.605156 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5880689-08fd-4b49-a06b-ab5ecab0d2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:17.605504 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5880689-08fd-4b49-a06b-ab5ecab0d2d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:17.606293 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5880689-08fd-4b49-a06b-ab5ecab0d2d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:17.608284 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5880689-08fd-4b49-a06b-ab5ecab0d2d3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:17.631637 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:37:17.640324 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1254/2524] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:37:17 2022] POST /identity/v3/projects => generated 389 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:37:17.644449 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2263a-5970-43d7-ba03-f0710f7d642b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:17.654501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2263a-5970-43d7-ba03-f0710f7d642b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:17.713208 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f19d858-564d-4198-85c8-781e644c9603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:17.714174 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19d858-564d-4198-85c8-781e644c9603 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:17.714464 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19d858-564d-4198-85c8-781e644c9603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:17.714746 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19d858-564d-4198-85c8-781e644c9603 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:17.715433 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f19d858-564d-4198-85c8-781e644c9603 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:17.717398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f19d858-564d-4198-85c8-781e644c9603 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:17.739859 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1271/2525] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:37:17 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:37:17.744031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5880689-08fd-4b49-a06b-ab5ecab0d2d3 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:17.756681 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5880689-08fd-4b49-a06b-ab5ecab0d2d3 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:17.761915 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6b4cf-efb1-43ef-8ec1-90abcdd2960a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:17.763066 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6b4cf-efb1-43ef-8ec1-90abcdd2960a admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:17.763556 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6b4cf-efb1-43ef-8ec1-90abcdd2960a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:17.764007 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6b4cf-efb1-43ef-8ec1-90abcdd2960a admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:17.764850 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6b4cf-efb1-43ef-8ec1-90abcdd2960a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:17.766494 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6b4cf-efb1-43ef-8ec1-90abcdd2960a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:17.778196 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1255/2526] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:37:17.782224 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f19d858-564d-4198-85c8-781e644c9603 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:17.793126 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f19d858-564d-4198-85c8-781e644c9603 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:17.797707 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1377523b-564e-48ff-82d5-61be1e3b423c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:17.798357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377523b-564e-48ff-82d5-61be1e3b423c admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:17.798639 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377523b-564e-48ff-82d5-61be1e3b423c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:17.799104 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377523b-564e-48ff-82d5-61be1e3b423c admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:17.799830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1377523b-564e-48ff-82d5-61be1e3b423c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:17.801415 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1377523b-564e-48ff-82d5-61be1e3b423c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:17.811227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1272/2527] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:37:17.815392 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6b4cf-efb1-43ef-8ec1-90abcdd2960a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:17.825572 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6b4cf-efb1-43ef-8ec1-90abcdd2960a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:17.830842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8312151-0c53-4eed-ab78-9d6a91cb6e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:17.831798 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8312151-0c53-4eed-ab78-9d6a91cb6e41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:17.832085 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8312151-0c53-4eed-ab78-9d6a91cb6e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:17.832362 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8312151-0c53-4eed-ab78-9d6a91cb6e41 admin admin] PATH_INFO: `/v3/projects/e6fcdd9fd0384c2f9088d340b897ee8b/users/5063ce7a59fa4fc1ac9e7417b85791a7/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:17.847898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8312151-0c53-4eed-ab78-9d6a91cb6e41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6fcdd9fd0384c2f9088d340b897ee8b, user_id=5063ce7a59fa4fc1ac9e7417b85791a7, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:17.850659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8312151-0c53-4eed-ab78-9d6a91cb6e41 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:17.863732 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1256/2528] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:37:17 2022] PUT /identity/v3/projects/e6fcdd9fd0384c2f9088d340b897ee8b/users/5063ce7a59fa4fc1ac9e7417b85791a7/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:37:17.926998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a837d9-b56a-4b77-a8f7-eba6a32d7fd1 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:17.927292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a837d9-b56a-4b77-a8f7-eba6a32d7fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:17.927570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a837d9-b56a-4b77-a8f7-eba6a32d7fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:17.953351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e6a837d9-b56a-4b77-a8f7-eba6a32d7fd1 None None] MFA Rules not processed for user `5063ce7a59fa4fc1ac9e7417b85791a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:37:17.954597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e6a837d9-b56a-4b77-a8f7-eba6a32d7fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:18.019176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1273/2529] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:37:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:37:18.030007 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8312151-0c53-4eed-ab78-9d6a91cb6e41 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:18.031784 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-c8312151-0c53-4eed-ab78-9d6a91cb6e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:18.092070 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8312151-0c53-4eed-ab78-9d6a91cb6e41 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:18.095918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6dfe71-5d9c-426b-ba4d-578ace20750e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:18.096637 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6dfe71-5d9c-426b-ba4d-578ace20750e service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:18.096926 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6dfe71-5d9c-426b-ba4d-578ace20750e service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:18.097207 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6dfe71-5d9c-426b-ba4d-578ace20750e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:18.106218 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6dfe71-5d9c-426b-ba4d-578ace20750e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:18.107712 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6dfe71-5d9c-426b-ba4d-578ace20750e service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:18.117968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1257/2530] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:37:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:37:28.015678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:37:28.113967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:37:37.872914 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab83dc3-7399-4b89-a64c-5980ea023e91 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:37.873373 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab83dc3-7399-4b89-a64c-5980ea023e91 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:37.873565 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab83dc3-7399-4b89-a64c-5980ea023e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:37.891627 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-dab83dc3-7399-4b89-a64c-5980ea023e91 None None] MFA Rules not processed for user `4ddcf6306e704fe9ad5ed8779178ae67`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:37:37.892848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-dab83dc3-7399-4b89-a64c-5980ea023e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:37.949913 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1274/2531] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 30 18:37:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:37:37.999834 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6dfe71-5d9c-426b-ba4d-578ace20750e service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:38.001264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2d6dfe71-5d9c-426b-ba4d-578ace20750e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:38.081738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6dfe71-5d9c-426b-ba4d-578ace20750e service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:38.087198 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe08c7d-25bb-4273-bf6a-a45c8fbc291f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bd521b9e93f4b3b8af64d46f36c019e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:38.087887 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe08c7d-25bb-4273-bf6a-a45c8fbc291f service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:38.088109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe08c7d-25bb-4273-bf6a-a45c8fbc291f service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:38.088397 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe08c7d-25bb-4273-bf6a-a45c8fbc291f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:38.098217 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe08c7d-25bb-4273-bf6a-a45c8fbc291f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:38.099889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe08c7d-25bb-4273-bf6a-a45c8fbc291f service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:38.108467 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1258/2532] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 30 18:37:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 18:37:42.650384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab83dc3-7399-4b89-a64c-5980ea023e91 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:42.752948 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab83dc3-7399-4b89-a64c-5980ea023e91 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:42.759633 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8d9cb-efda-4dab-86ab-f34c426042d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:42.760533 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8d9cb-efda-4dab-86ab-f34c426042d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:42.760825 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8d9cb-efda-4dab-86ab-f34c426042d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:42.761106 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8d9cb-efda-4dab-86ab-f34c426042d2 admin admin] PATH_INFO: `/v3/users/5063ce7a59fa4fc1ac9e7417b85791a7` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:42.762968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8d9cb-efda-4dab-86ab-f34c426042d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5063ce7a59fa4fc1ac9e7417b85791a7)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:42.765283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8d9cb-efda-4dab-86ab-f34c426042d2 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:42.839307 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bcd8d9cb-efda-4dab-86ab-f34c426042d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5063ce7a59fa4fc1ac9e7417b85791a7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:37:42.861963 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bcd8d9cb-efda-4dab-86ab-f34c426042d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5063ce7a59fa4fc1ac9e7417b85791a7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:37:42.862853 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bcd8d9cb-efda-4dab-86ab-f34c426042d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5063ce7a59fa4fc1ac9e7417b85791a7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:37:42.876520 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bcd8d9cb-efda-4dab-86ab-f34c426042d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5063ce7a59fa4fc1ac9e7417b85791a7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:37:42.877230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-bcd8d9cb-efda-4dab-86ab-f34c426042d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5063ce7a59fa4fc1ac9e7417b85791a7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:37:42.894587 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1275/2533] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:37:42 2022] DELETE /identity/v3/users/5063ce7a59fa4fc1ac9e7417b85791a7 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:37:42.899340 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7b11b-f2e5-46b8-98b8-e156d753d57a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:42.900270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7b11b-f2e5-46b8-98b8-e156d753d57a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:42.900715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7b11b-f2e5-46b8-98b8-e156d753d57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:42.919657 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b2d7b11b-f2e5-46b8-98b8-e156d753d57a None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:37:42.920903 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b2d7b11b-f2e5-46b8-98b8-e156d753d57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:43.013305 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1259/2534] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:37:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:37:43.027194 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8d9cb-efda-4dab-86ab-f34c426042d2 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:43.030729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bcd8d9cb-efda-4dab-86ab-f34c426042d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:43.075064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:37:43.117185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8d9cb-efda-4dab-86ab-f34c426042d2 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:43.121361 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc2daed-2ef7-4ca6-85f5-757c1ec89d81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:43.122154 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc2daed-2ef7-4ca6-85f5-757c1ec89d81 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:43.122521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc2daed-2ef7-4ca6-85f5-757c1ec89d81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:43.122870 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc2daed-2ef7-4ca6-85f5-757c1ec89d81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:43.133972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc2daed-2ef7-4ca6-85f5-757c1ec89d81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:43.135561 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc2daed-2ef7-4ca6-85f5-757c1ec89d81 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:43.163338 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1276/2535] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:37:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:37:43.574086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7b11b-f2e5-46b8-98b8-e156d753d57a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:43.576304 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b2d7b11b-f2e5-46b8-98b8-e156d753d57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:43.618292 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7b11b-f2e5-46b8-98b8-e156d753d57a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:43.624095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee567c-8225-4fc5-a65b-c79a5ba0bca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:43.625097 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee567c-8225-4fc5-a65b-c79a5ba0bca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:43.625489 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee567c-8225-4fc5-a65b-c79a5ba0bca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:43.625901 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee567c-8225-4fc5-a65b-c79a5ba0bca9 admin admin] PATH_INFO: `/v3/projects/e6fcdd9fd0384c2f9088d340b897ee8b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:43.628170 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee567c-8225-4fc5-a65b-c79a5ba0bca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6fcdd9fd0384c2f9088d340b897ee8b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:43.630309 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee567c-8225-4fc5-a65b-c79a5ba0bca9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:43.741223 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-47ee567c-8225-4fc5-a65b-c79a5ba0bca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6fcdd9fd0384c2f9088d340b897ee8b', 'request_id': 'req-47ee567c-8225-4fc5-a65b-c79a5ba0bca9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:37:43.753253 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-47ee567c-8225-4fc5-a65b-c79a5ba0bca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6fcdd9fd0384c2f9088d340b897ee8b', 'request_id': 'req-47ee567c-8225-4fc5-a65b-c79a5ba0bca9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:37:43.762927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-47ee567c-8225-4fc5-a65b-c79a5ba0bca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6fcdd9fd0384c2f9088d340b897ee8b', 'request_id': 'req-47ee567c-8225-4fc5-a65b-c79a5ba0bca9'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:37:43.779662 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-47ee567c-8225-4fc5-a65b-c79a5ba0bca9 admin admin] The token cache is being invalidate because project e6fcdd9fd0384c2f9088d340b897ee8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:37:43.779957 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-47ee567c-8225-4fc5-a65b-c79a5ba0bca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:37:43.780627 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-47ee567c-8225-4fc5-a65b-c79a5ba0bca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:37:43.787565 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1260/2536] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:37:43 2022] DELETE /identity/v3/projects/e6fcdd9fd0384c2f9088d340b897ee8b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:37:43.789672 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07ceda-3de2-40f0-9bfb-e2e39a7fe483 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:43.790493 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07ceda-3de2-40f0-9bfb-e2e39a7fe483 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:43.791416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07ceda-3de2-40f0-9bfb-e2e39a7fe483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:43.812695 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0c07ceda-3de2-40f0-9bfb-e2e39a7fe483 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:37:43.814355 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0c07ceda-3de2-40f0-9bfb-e2e39a7fe483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:43.904098 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1277/2537] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:37:43.912109 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fb291-f6ac-4682-8edc-1e7170910b93 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:43.912671 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fb291-f6ac-4682-8edc-1e7170910b93 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:43.913012 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fb291-f6ac-4682-8edc-1e7170910b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:43.937782 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f57fb291-f6ac-4682-8edc-1e7170910b93 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:37:43.940288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f57fb291-f6ac-4682-8edc-1e7170910b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:43.952262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1261/2538] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:37:43.956114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07ceda-3de2-40f0-9bfb-e2e39a7fe483 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:43.970292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07ceda-3de2-40f0-9bfb-e2e39a7fe483 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:43.979880 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa3089a-2312-48ce-84bb-0992dda3cfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:43.980720 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3089a-2312-48ce-84bb-0992dda3cfea admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:43.981017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3089a-2312-48ce-84bb-0992dda3cfea admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:43.981299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3089a-2312-48ce-84bb-0992dda3cfea admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:43.982180 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa3089a-2312-48ce-84bb-0992dda3cfea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:37:43.982499 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa3089a-2312-48ce-84bb-0992dda3cfea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:43.984292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa3089a-2312-48ce-84bb-0992dda3cfea admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:44.001755 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1278/2539] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:37:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:37:44.006632 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fb291-f6ac-4682-8edc-1e7170910b93 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:44.021856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fb291-f6ac-4682-8edc-1e7170910b93 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:44.031305 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab861d-0f5b-4e2e-8396-9acf92959aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:44.032050 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab861d-0f5b-4e2e-8396-9acf92959aaf admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:44.032497 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab861d-0f5b-4e2e-8396-9acf92959aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:44.033099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab861d-0f5b-4e2e-8396-9acf92959aaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:44.035022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab861d-0f5b-4e2e-8396-9acf92959aaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:44.039543 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab861d-0f5b-4e2e-8396-9acf92959aaf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:44.043537 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:37:44.088237 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1262/2540] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:37:44 2022] POST /identity/v3/projects => generated 385 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:37:44.089424 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa3089a-2312-48ce-84bb-0992dda3cfea admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:44.108833 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa3089a-2312-48ce-84bb-0992dda3cfea admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:44.220753 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36031ad-070a-4669-8635-11521a1ddc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:44.221795 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36031ad-070a-4669-8635-11521a1ddc42 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:44.222265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36031ad-070a-4669-8635-11521a1ddc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:44.222831 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36031ad-070a-4669-8635-11521a1ddc42 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:44.224519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36031ad-070a-4669-8635-11521a1ddc42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:44.226832 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36031ad-070a-4669-8635-11521a1ddc42 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:44.256380 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1279/2541] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:37:44 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:37:44.257610 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab861d-0f5b-4e2e-8396-9acf92959aaf admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:44.269033 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab861d-0f5b-4e2e-8396-9acf92959aaf admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:44.275263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf43554-1b11-4528-a1fa-f47f5bd6c6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:44.275470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf43554-1b11-4528-a1fa-f47f5bd6c6b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:44.275666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf43554-1b11-4528-a1fa-f47f5bd6c6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:44.275949 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf43554-1b11-4528-a1fa-f47f5bd6c6b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:44.276697 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf43554-1b11-4528-a1fa-f47f5bd6c6b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:44.278523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf43554-1b11-4528-a1fa-f47f5bd6c6b6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:44.294686 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36031ad-070a-4669-8635-11521a1ddc42 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:44.303056 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1263/2542] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:37:44.307782 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36031ad-070a-4669-8635-11521a1ddc42 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:44.314369 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f64bf-3a25-4a72-9bd7-b162340429d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:44.315049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f64bf-3a25-4a72-9bd7-b162340429d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:44.315610 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f64bf-3a25-4a72-9bd7-b162340429d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:44.315956 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f64bf-3a25-4a72-9bd7-b162340429d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:44.316708 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f64bf-3a25-4a72-9bd7-b162340429d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:44.319416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f64bf-3a25-4a72-9bd7-b162340429d5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:44.332435 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1280/2543] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:37:44.337147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf43554-1b11-4528-a1fa-f47f5bd6c6b6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:44.349061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf43554-1b11-4528-a1fa-f47f5bd6c6b6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:44.354056 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8316bf9-10a2-4c5c-8a41-65697ccd285f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:44.354950 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8316bf9-10a2-4c5c-8a41-65697ccd285f admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:44.355235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8316bf9-10a2-4c5c-8a41-65697ccd285f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:44.355517 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8316bf9-10a2-4c5c-8a41-65697ccd285f admin admin] PATH_INFO: `/v3/projects/666edd3825ba4e32ac180ed230d93e85/users/784fdfd56b3d4ae4b6f68b66c4f06ea5/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:44.393356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8316bf9-10a2-4c5c-8a41-65697ccd285f admin admin] RBAC: Authorizing `identity:create_grant(project_id=666edd3825ba4e32ac180ed230d93e85, user_id=784fdfd56b3d4ae4b6f68b66c4f06ea5, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:44.396660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8316bf9-10a2-4c5c-8a41-65697ccd285f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:44.436880 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6d489-2831-4408-ad58-d437358d9972 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:44.437729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6d489-2831-4408-ad58-d437358d9972 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:44.438021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6d489-2831-4408-ad58-d437358d9972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:44.439444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1264/2544] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:37:44 2022] PUT /identity/v3/projects/666edd3825ba4e32ac180ed230d93e85/users/784fdfd56b3d4ae4b6f68b66c4f06ea5/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:37:44.460341 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d7f6d489-2831-4408-ad58-d437358d9972 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:37:44.461611 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d7f6d489-2831-4408-ad58-d437358d9972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:44.523745 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:37:44.576871 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1281/2545] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:37:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:37:44.586093 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8316bf9-10a2-4c5c-8a41-65697ccd285f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:44.587864 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f8316bf9-10a2-4c5c-8a41-65697ccd285f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:44.705117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8316bf9-10a2-4c5c-8a41-65697ccd285f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:44.718206 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77406f9-856f-4228-b5dc-0a3783f3d50b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:44.719429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77406f9-856f-4228-b5dc-0a3783f3d50b service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:44.720266 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77406f9-856f-4228-b5dc-0a3783f3d50b service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:44.720701 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77406f9-856f-4228-b5dc-0a3783f3d50b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:44.738803 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77406f9-856f-4228-b5dc-0a3783f3d50b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:44.742699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77406f9-856f-4228-b5dc-0a3783f3d50b service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:44.772970 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1265/2546] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:37:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:37:50.013767 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07835173-1a46-4326-a544-bff0e32d748f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:50.014245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07835173-1a46-4326-a544-bff0e32d748f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:50.014329 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07835173-1a46-4326-a544-bff0e32d748f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:50.048441 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-07835173-1a46-4326-a544-bff0e32d748f None None] MFA Rules not processed for user `784fdfd56b3d4ae4b6f68b66c4f06ea5`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:37:50.049804 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-07835173-1a46-4326-a544-bff0e32d748f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:50.129467 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1282/2547] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:37:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:37:50.134480 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77406f9-856f-4228-b5dc-0a3783f3d50b service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:50.144877 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77406f9-856f-4228-b5dc-0a3783f3d50b service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:50.160559 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6a503-63ca-4bad-962a-14c3778553af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:50.162289 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6a503-63ca-4bad-962a-14c3778553af admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:50.162660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6a503-63ca-4bad-962a-14c3778553af admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:50.163036 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6a503-63ca-4bad-962a-14c3778553af admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:50.164122 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6a503-63ca-4bad-962a-14c3778553af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:50.166761 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6a503-63ca-4bad-962a-14c3778553af admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:50.197918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1266/2548] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:37:50 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:37:50.201903 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07835173-1a46-4326-a544-bff0e32d748f None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:50.216083 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07835173-1a46-4326-a544-bff0e32d748f None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:50.276647 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cb465-b4ad-48cb-bce4-080d439fad11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:50.277324 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cb465-b4ad-48cb-bce4-080d439fad11 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:50.277633 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cb465-b4ad-48cb-bce4-080d439fad11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:50.277915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cb465-b4ad-48cb-bce4-080d439fad11 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:50.278615 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243cb465-b4ad-48cb-bce4-080d439fad11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:50.280558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243cb465-b4ad-48cb-bce4-080d439fad11 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:50.302595 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1283/2549] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:37:50 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:37:50.306750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6a503-63ca-4bad-962a-14c3778553af admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:50.318037 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6a503-63ca-4bad-962a-14c3778553af admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:50.322850 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6904508-6e8a-4014-8e16-dc1789c1d1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:50.323523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6904508-6e8a-4014-8e16-dc1789c1d1bb admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:50.323813 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6904508-6e8a-4014-8e16-dc1789c1d1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:50.324095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6904508-6e8a-4014-8e16-dc1789c1d1bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:50.324829 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6904508-6e8a-4014-8e16-dc1789c1d1bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:50.326486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6904508-6e8a-4014-8e16-dc1789c1d1bb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:50.338033 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1267/2550] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:37:50.342616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cb465-b4ad-48cb-bce4-080d439fad11 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:50.353855 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cb465-b4ad-48cb-bce4-080d439fad11 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:50.358726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa78be-d393-47b2-9150-ae5639ce7cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:50.359619 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa78be-d393-47b2-9150-ae5639ce7cfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:50.359906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa78be-d393-47b2-9150-ae5639ce7cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:50.360210 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa78be-d393-47b2-9150-ae5639ce7cfe admin admin] PATH_INFO: `/v3/projects/22ccfe9956dd4b62b25386e0b09abc0b/users/4ae86a79a2de4ff7b9d5df3a3bfcbd14/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:50.376883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aa78be-d393-47b2-9150-ae5639ce7cfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=22ccfe9956dd4b62b25386e0b09abc0b, user_id=4ae86a79a2de4ff7b9d5df3a3bfcbd14, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:50.379651 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aa78be-d393-47b2-9150-ae5639ce7cfe admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:50.394922 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1284/2551] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:37:50 2022] PUT /identity/v3/projects/22ccfe9956dd4b62b25386e0b09abc0b/users/4ae86a79a2de4ff7b9d5df3a3bfcbd14/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:37:50.399344 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6904508-6e8a-4014-8e16-dc1789c1d1bb admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:50.409834 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6904508-6e8a-4014-8e16-dc1789c1d1bb admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:50.471614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea0a12-4e0f-41ed-b1a7-11383a01fb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:50.472431 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea0a12-4e0f-41ed-b1a7-11383a01fb1a admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:50.472715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea0a12-4e0f-41ed-b1a7-11383a01fb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:50.473127 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea0a12-4e0f-41ed-b1a7-11383a01fb1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:50.474703 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eea0a12-4e0f-41ed-b1a7-11383a01fb1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:50.476359 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eea0a12-4e0f-41ed-b1a7-11383a01fb1a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:50.487367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1268/2552] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:37:50.492289 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa78be-d393-47b2-9150-ae5639ce7cfe admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:50.503627 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa78be-d393-47b2-9150-ae5639ce7cfe admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:50.508794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79576d-dfec-46f5-b26f-511da87fd308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:50.509957 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79576d-dfec-46f5-b26f-511da87fd308 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:50.510315 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79576d-dfec-46f5-b26f-511da87fd308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:50.510667 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79576d-dfec-46f5-b26f-511da87fd308 admin admin] PATH_INFO: `/v3/projects/22ccfe9956dd4b62b25386e0b09abc0b/users/4ae86a79a2de4ff7b9d5df3a3bfcbd14/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:50.512788 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac79576d-dfec-46f5-b26f-511da87fd308 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22ccfe9956dd4b62b25386e0b09abc0b, user_id=4ae86a79a2de4ff7b9d5df3a3bfcbd14, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:50.515570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac79576d-dfec-46f5-b26f-511da87fd308 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:50.527066 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1285/2553] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:37:50 2022] PUT /identity/v3/projects/22ccfe9956dd4b62b25386e0b09abc0b/users/4ae86a79a2de4ff7b9d5df3a3bfcbd14/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:37:55.353378 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf8ef1-26b5-4e9f-ba03-d74d983c7ee8 None None] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:55.356152 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf8ef1-26b5-4e9f-ba03-d74d983c7ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:55.356152 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf8ef1-26b5-4e9f-ba03-d74d983c7ee8 None None] PATH_INFO: `/` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:55.356152 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1269/2554] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 30 18:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:37:55.367368 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da1154-e365-4926-ae7d-fe381e503f2f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:55.367897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da1154-e365-4926-ae7d-fe381e503f2f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:55.367897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da1154-e365-4926-ae7d-fe381e503f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:55.394240 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-98da1154-e365-4926-ae7d-fe381e503f2f None None] MFA Rules not processed for user `b8751a4e92174d18b7b138392d2579a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:37:55.397542 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-98da1154-e365-4926-ae7d-fe381e503f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:55.413873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:37:55.511057 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1286/2555] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 30 18:37:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:37:55.525024 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf8ef1-26b5-4e9f-ba03-d74d983c7ee8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:55.526811 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a0bf8ef1-26b5-4e9f-ba03-d74d983c7ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:55.570962 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:37:55.615155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf8ef1-26b5-4e9f-ba03-d74d983c7ee8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:55.619143 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3895b-029d-4f1e-ad7b-f04d7d57e8c2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37a9fea702cd405ea9fc14ec2c689648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:55.619803 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3895b-029d-4f1e-ad7b-f04d7d57e8c2 service glance] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:55.620090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3895b-029d-4f1e-ad7b-f04d7d57e8c2 service glance] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:55.620372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3895b-029d-4f1e-ad7b-f04d7d57e8c2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:55.633224 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3895b-029d-4f1e-ad7b-f04d7d57e8c2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:55.636119 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3895b-029d-4f1e-ad7b-f04d7d57e8c2 service glance] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:55.667915 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1270/2556] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:37:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:37:55.748582 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dea7ba-4aed-4c70-9b50-a8907c11e3c2 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:55.749037 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dea7ba-4aed-4c70-9b50-a8907c11e3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:55.749398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dea7ba-4aed-4c70-9b50-a8907c11e3c2 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:55.751065 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1287/2557] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 30 18:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:37:55.757557 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18052ade-0a63-4b81-a711-ba6ea3a92eba None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:55.757859 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18052ade-0a63-4b81-a711-ba6ea3a92eba None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:55.758150 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18052ade-0a63-4b81-a711-ba6ea3a92eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:55.787336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-18052ade-0a63-4b81-a711-ba6ea3a92eba None None] MFA Rules not processed for user `1d8c54102fb340859ed0acc3189fe376`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:37:55.789193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-18052ade-0a63-4b81-a711-ba6ea3a92eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:55.828022 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1155708-4492-4415-9c44-08fb1aa0e5c0 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:55.828316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1155708-4492-4415-9c44-08fb1aa0e5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:55.828602 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1155708-4492-4415-9c44-08fb1aa0e5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:55.860079 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1271/2558] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 30 18:37:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:37:55.861797 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b1155708-4492-4415-9c44-08fb1aa0e5c0 None None] MFA Rules not processed for user `4ae86a79a2de4ff7b9d5df3a3bfcbd14`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:37:55.863027 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b1155708-4492-4415-9c44-08fb1aa0e5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:55.875136 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18052ade-0a63-4b81-a711-ba6ea3a92eba None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:55.876513 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-18052ade-0a63-4b81-a711-ba6ea3a92eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:55.887627 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18052ade-0a63-4b81-a711-ba6ea3a92eba None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:55.891715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cedac47-eed9-42f8-afcd-6cbd0f8c39f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:55.892356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cedac47-eed9-42f8-afcd-6cbd0f8c39f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:55.892636 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cedac47-eed9-42f8-afcd-6cbd0f8c39f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:55.892911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cedac47-eed9-42f8-afcd-6cbd0f8c39f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:55.901103 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cedac47-eed9-42f8-afcd-6cbd0f8c39f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:55.902856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cedac47-eed9-42f8-afcd-6cbd0f8c39f7 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:55.912180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1272/2559] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:37:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:37:55.945353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1288/2560] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:37:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:37:55.958037 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cedac47-eed9-42f8-afcd-6cbd0f8c39f7 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:55.968741 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cedac47-eed9-42f8-afcd-6cbd0f8c39f7 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:55.972663 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f154d2-0a3b-4852-9ea0-53ed7caafb70 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37a9fea702cd405ea9fc14ec2c689648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:55.973291 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f154d2-0a3b-4852-9ea0-53ed7caafb70 service glance] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:55.973572 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f154d2-0a3b-4852-9ea0-53ed7caafb70 service glance] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:55.974665 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f154d2-0a3b-4852-9ea0-53ed7caafb70 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:56.006695 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6366888-8818-456f-af71-a11407fdb010 None None] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:56.006902 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6366888-8818-456f-af71-a11407fdb010 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:56.007184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6366888-8818-456f-af71-a11407fdb010 None None] PATH_INFO: `/` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:56.008754 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1289/2561] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 30 18:37:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 18:37:56.015996 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34169c4-b837-4ef2-9f7c-95f4a1388003 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:56.016571 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34169c4-b837-4ef2-9f7c-95f4a1388003 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:56.016853 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34169c4-b837-4ef2-9f7c-95f4a1388003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:56.039928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b34169c4-b837-4ef2-9f7c-95f4a1388003 None None] MFA Rules not processed for user `b8751a4e92174d18b7b138392d2579a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:37:56.041501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b34169c4-b837-4ef2-9f7c-95f4a1388003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:56.053484 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1290/2562] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 30 18:37:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:37:56.066681 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f154d2-0a3b-4852-9ea0-53ed7caafb70 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:56.067324 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34169c4-b837-4ef2-9f7c-95f4a1388003 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:56.068524 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f154d2-0a3b-4852-9ea0-53ed7caafb70 service glance] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:56.077886 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34169c4-b837-4ef2-9f7c-95f4a1388003 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:56.080157 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1273/2563] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:37:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:37:56.082262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb0654-9861-4df8-aede-510f693f104a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:56.082910 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb0654-9861-4df8-aede-510f693f104a service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:56.083895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb0654-9861-4df8-aede-510f693f104a service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:56.084177 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb0654-9861-4df8-aede-510f693f104a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:56.094989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb0654-9861-4df8-aede-510f693f104a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:56.096440 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb0654-9861-4df8-aede-510f693f104a service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:56.109351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1291/2564] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:37:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:37:56.214686 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af261937-aaf0-4a7e-bd0e-65c0bed24931 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:56.214686 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af261937-aaf0-4a7e-bd0e-65c0bed24931 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:56.215067 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af261937-aaf0-4a7e-bd0e-65c0bed24931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:56.233377 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-af261937-aaf0-4a7e-bd0e-65c0bed24931 None None] MFA Rules not processed for user `4ddcf6306e704fe9ad5ed8779178ae67`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:37:56.234641 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-af261937-aaf0-4a7e-bd0e-65c0bed24931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:56.288822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:37:56.300118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1274/2565] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 30 18:37:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:37:56.314624 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb0654-9861-4df8-aede-510f693f104a service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:56.316177 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-a9cb0654-9861-4df8-aede-510f693f104a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:56.419466 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb0654-9861-4df8-aede-510f693f104a service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:56.427471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88bb013-ad7f-4ab9-84d5-e044a3cc02dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bd521b9e93f4b3b8af64d46f36c019e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:56.428118 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88bb013-ad7f-4ab9-84d5-e044a3cc02dd service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:56.429925 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88bb013-ad7f-4ab9-84d5-e044a3cc02dd service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:56.429925 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88bb013-ad7f-4ab9-84d5-e044a3cc02dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:56.440972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88bb013-ad7f-4ab9-84d5-e044a3cc02dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:56.444234 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88bb013-ad7f-4ab9-84d5-e044a3cc02dd service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:56.455912 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1292/2566] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 30 18:37:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 30 18:37:58.327970 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80891df1-8804-4015-ac08-44d768220e6c None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:58.328523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80891df1-8804-4015-ac08-44d768220e6c None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:58.328523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80891df1-8804-4015-ac08-44d768220e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:58.354337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-80891df1-8804-4015-ac08-44d768220e6c None None] MFA Rules not processed for user `4ddcf6306e704fe9ad5ed8779178ae67`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:37:58.355619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-80891df1-8804-4015-ac08-44d768220e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:58.366784 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1275/2567] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 30 18:37:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:37:58.387350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88bb013-ad7f-4ab9-84d5-e044a3cc02dd service swift] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:58.393418 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:37:58.401407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88bb013-ad7f-4ab9-84d5-e044a3cc02dd service swift] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:58.407756 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb149a6-f42c-4d24-9110-52bdb791cb47 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bd521b9e93f4b3b8af64d46f36c019e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:58.408509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb149a6-f42c-4d24-9110-52bdb791cb47 service swift] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:58.408876 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb149a6-f42c-4d24-9110-52bdb791cb47 service swift] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:58.409235 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb149a6-f42c-4d24-9110-52bdb791cb47 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:58.420153 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb149a6-f42c-4d24-9110-52bdb791cb47 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:58.420745 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb149a6-f42c-4d24-9110-52bdb791cb47 service swift] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:58.431047 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1293/2568] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 30 18:37:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 30 18:38:07.157430 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571005a-8762-4908-89ce-baa5d5bb4a32 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:38:07.158086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571005a-8762-4908-89ce-baa5d5bb4a32 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:38:07.158086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571005a-8762-4908-89ce-baa5d5bb4a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:38:07.179490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2571005a-8762-4908-89ce-baa5d5bb4a32 None None] MFA Rules not processed for user `784fdfd56b3d4ae4b6f68b66c4f06ea5`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:38:07.180972 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2571005a-8762-4908-89ce-baa5d5bb4a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:38:07.199367 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1276/2569] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:38:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:38:08.426332 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:38:15.972317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2346e-d66e-4697-93ee-d26eb5cb3d22 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:38:15.972973 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2346e-d66e-4697-93ee-d26eb5cb3d22 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:38:15.972973 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2346e-d66e-4697-93ee-d26eb5cb3d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:38:15.991583 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-86f2346e-d66e-4697-93ee-d26eb5cb3d22 None None] MFA Rules not processed for user `4ddcf6306e704fe9ad5ed8779178ae67`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:38:15.992911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-86f2346e-d66e-4697-93ee-d26eb5cb3d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:38:16.000039 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1294/2570] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 30 18:38:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:38:16.012883 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571005a-8762-4908-89ce-baa5d5bb4a32 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:38:16.025933 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571005a-8762-4908-89ce-baa5d5bb4a32 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:38:16.031299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22dcf7-e9d1-475e-8a0b-0a07847c19f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bd521b9e93f4b3b8af64d46f36c019e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:38:16.031940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22dcf7-e9d1-475e-8a0b-0a07847c19f6 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:38:16.032220 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22dcf7-e9d1-475e-8a0b-0a07847c19f6 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:38:16.032493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22dcf7-e9d1-475e-8a0b-0a07847c19f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:38:16.041678 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe22dcf7-e9d1-475e-8a0b-0a07847c19f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:38:16.043217 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe22dcf7-e9d1-475e-8a0b-0a07847c19f6 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:38:16.051672 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1277/2571] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 30 18:38:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 18:38:25.621888 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2346e-d66e-4697-93ee-d26eb5cb3d22 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:38:25.631181 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2346e-d66e-4697-93ee-d26eb5cb3d22 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:38:25.635310 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fa2bd-594a-49c4-9986-128160a09426 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:38:25.635934 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fa2bd-594a-49c4-9986-128160a09426 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:38:25.636216 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fa2bd-594a-49c4-9986-128160a09426 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:38:25.636492 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fa2bd-594a-49c4-9986-128160a09426 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:38:25.638271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-894fa2bd-594a-49c4-9986-128160a09426 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:38:25.651883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894fa2bd-594a-49c4-9986-128160a09426 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:38:25.653294 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894fa2bd-594a-49c4-9986-128160a09426 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:38:25.663942 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1295/2572] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:38:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:38:26.048635 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=13 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:38:28.625566 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22dcf7-e9d1-475e-8a0b-0a07847c19f6 service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:38:28.635353 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22dcf7-e9d1-475e-8a0b-0a07847c19f6 service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:38:28.638921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc8a31-3578-40ec-a6c8-09589e8ccfa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:38:28.639556 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc8a31-3578-40ec-a6c8-09589e8ccfa6 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:38:28.639842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc8a31-3578-40ec-a6c8-09589e8ccfa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:38:28.640122 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc8a31-3578-40ec-a6c8-09589e8ccfa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:38:28.641948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3abc8a31-3578-40ec-a6c8-09589e8ccfa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:38:28.656415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc8a31-3578-40ec-a6c8-09589e8ccfa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:38:28.657928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc8a31-3578-40ec-a6c8-09589e8ccfa6 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:38:28.673874 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1278/2573] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:38:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:38:30.411580 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9304c-f5a4-4ea3-a01d-24c70db424b6 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:38:30.412062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9304c-f5a4-4ea3-a01d-24c70db424b6 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:38:30.412146 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9304c-f5a4-4ea3-a01d-24c70db424b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:38:30.431724 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-28a9304c-f5a4-4ea3-a01d-24c70db424b6 None None] MFA Rules not processed for user `4ddcf6306e704fe9ad5ed8779178ae67`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:38:30.432959 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-28a9304c-f5a4-4ea3-a01d-24c70db424b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:38:30.440375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1296/2574] 10.222.0.33 () {58 vars in 967 bytes} [Thu Jun 30 18:38:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:38:30.454424 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc8a31-3578-40ec-a6c8-09589e8ccfa6 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:38:30.466294 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc8a31-3578-40ec-a6c8-09589e8ccfa6 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:38:30.471808 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0b26f-ea92-4c8a-a081-cadc72a5d273 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bd521b9e93f4b3b8af64d46f36c019e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:38:30.472776 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0b26f-ea92-4c8a-a081-cadc72a5d273 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:38:30.473261 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0b26f-ea92-4c8a-a081-cadc72a5d273 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:38:30.473767 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0b26f-ea92-4c8a-a081-cadc72a5d273 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:38:30.498288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0b26f-ea92-4c8a-a081-cadc72a5d273 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:38:30.500179 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0b26f-ea92-4c8a-a081-cadc72a5d273 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:38:30.510528 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1279/2575] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 30 18:38:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 18:38:40.425488 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=12 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:38:40.505546 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=12 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:38:50.397657 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9304c-f5a4-4ea3-a01d-24c70db424b6 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:38:50.407087 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9304c-f5a4-4ea3-a01d-24c70db424b6 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:38:50.411224 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac8632-74e8-443b-ac24-db6669237801 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:38:50.411859 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac8632-74e8-443b-ac24-db6669237801 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:38:50.412145 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac8632-74e8-443b-ac24-db6669237801 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:38:50.412426 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac8632-74e8-443b-ac24-db6669237801 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:38:50.421191 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac8632-74e8-443b-ac24-db6669237801 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:38:50.422662 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac8632-74e8-443b-ac24-db6669237801 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:38:50.432570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1297/2576] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:38:50 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:39:00.428429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:20.623463 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0b26f-ea92-4c8a-a081-cadc72a5d273 service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:20.632753 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0b26f-ea92-4c8a-a081-cadc72a5d273 service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:20.636297 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb882a4-3ee1-4bdc-93d2-5dfd6e8e7183 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:20.637396 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb882a4-3ee1-4bdc-93d2-5dfd6e8e7183 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:20.637717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb882a4-3ee1-4bdc-93d2-5dfd6e8e7183 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:20.638001 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb882a4-3ee1-4bdc-93d2-5dfd6e8e7183 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:20.639743 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0bb882a4-3ee1-4bdc-93d2-5dfd6e8e7183 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:20.702465 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb882a4-3ee1-4bdc-93d2-5dfd6e8e7183 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:20.703897 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb882a4-3ee1-4bdc-93d2-5dfd6e8e7183 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:20.713720 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1280/2577] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:39:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:39:23.914612 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac8632-74e8-443b-ac24-db6669237801 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:23.917485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f5ac8632-74e8-443b-ac24-db6669237801 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:23.971401 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb882a4-3ee1-4bdc-93d2-5dfd6e8e7183 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:23.973480 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-0bb882a4-3ee1-4bdc-93d2-5dfd6e8e7183 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:23.987395 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb882a4-3ee1-4bdc-93d2-5dfd6e8e7183 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:23.992624 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e759c7d8-a028-43ce-a57b-249c0ed83443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:23.993906 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759c7d8-a028-43ce-a57b-249c0ed83443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:23.994262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759c7d8-a028-43ce-a57b-249c0ed83443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:23.995361 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759c7d8-a028-43ce-a57b-249c0ed83443 admin admin] PATH_INFO: `/v3/users/6ac4fc6cab4f47ea910ffdbed0a8e5b8` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:23.996624 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac8632-74e8-443b-ac24-db6669237801 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:23.996760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e759c7d8-a028-43ce-a57b-249c0ed83443 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ac4fc6cab4f47ea910ffdbed0a8e5b8)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:23.998841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e759c7d8-a028-43ce-a57b-249c0ed83443 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:24.002610 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feafaae-4e10-42bc-80d0-1252d8f962fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49325c1733174e0c98b3475a10c9b5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:24.003360 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feafaae-4e10-42bc-80d0-1252d8f962fc service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:24.003641 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feafaae-4e10-42bc-80d0-1252d8f962fc service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:24.003925 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feafaae-4e10-42bc-80d0-1252d8f962fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:24.005828 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-5feafaae-4e10-42bc-80d0-1252d8f962fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:24.019704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feafaae-4e10-42bc-80d0-1252d8f962fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:24.021332 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feafaae-4e10-42bc-80d0-1252d8f962fc service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:24.038601 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1298/2578] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:39:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:39:24.061189 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e759c7d8-a028-43ce-a57b-249c0ed83443 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ac4fc6cab4f47ea910ffdbed0a8e5b8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:24.061996 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e759c7d8-a028-43ce-a57b-249c0ed83443 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ac4fc6cab4f47ea910ffdbed0a8e5b8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:24.076788 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e759c7d8-a028-43ce-a57b-249c0ed83443 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ac4fc6cab4f47ea910ffdbed0a8e5b8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:24.084360 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e759c7d8-a028-43ce-a57b-249c0ed83443 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ac4fc6cab4f47ea910ffdbed0a8e5b8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:24.085038 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e759c7d8-a028-43ce-a57b-249c0ed83443 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ac4fc6cab4f47ea910ffdbed0a8e5b8'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:24.098881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1281/2579] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:39:23 2022] DELETE /identity/v3/users/6ac4fc6cab4f47ea910ffdbed0a8e5b8 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:24.273913 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feafaae-4e10-42bc-80d0-1252d8f962fc service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:24.286210 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feafaae-4e10-42bc-80d0-1252d8f962fc service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:24.344734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39a3fa-d074-46f9-870a-39cf27ed765c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:24.345499 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39a3fa-d074-46f9-870a-39cf27ed765c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:24.345822 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39a3fa-d074-46f9-870a-39cf27ed765c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:24.346102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39a3fa-d074-46f9-870a-39cf27ed765c admin admin] PATH_INFO: `/v3/projects/76384f04296b43b69e675de14cf1a55a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:24.347306 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39a3fa-d074-46f9-870a-39cf27ed765c admin admin] RBAC: Authorizing `identity:delete_project(project_id=76384f04296b43b69e675de14cf1a55a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:24.349236 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39a3fa-d074-46f9-870a-39cf27ed765c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:24.395226 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7b39a3fa-d074-46f9-870a-39cf27ed765c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76384f04296b43b69e675de14cf1a55a', 'request_id': 'req-7b39a3fa-d074-46f9-870a-39cf27ed765c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:24.401306 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7b39a3fa-d074-46f9-870a-39cf27ed765c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76384f04296b43b69e675de14cf1a55a', 'request_id': 'req-7b39a3fa-d074-46f9-870a-39cf27ed765c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:24.407119 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7b39a3fa-d074-46f9-870a-39cf27ed765c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76384f04296b43b69e675de14cf1a55a', 'request_id': 'req-7b39a3fa-d074-46f9-870a-39cf27ed765c'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:24.420422 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7b39a3fa-d074-46f9-870a-39cf27ed765c admin admin] The token cache is being invalidate because project 76384f04296b43b69e675de14cf1a55a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:39:24.420709 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7b39a3fa-d074-46f9-870a-39cf27ed765c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:24.421330 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7b39a3fa-d074-46f9-870a-39cf27ed765c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:24.422880 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1299/2580] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:39:24 2022] DELETE /identity/v3/projects/76384f04296b43b69e675de14cf1a55a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:24.427748 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6aa3dd-6913-428b-9412-0d3947f1e166 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:24.428041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6aa3dd-6913-428b-9412-0d3947f1e166 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:24.428322 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6aa3dd-6913-428b-9412-0d3947f1e166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:24.446392 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4f6aa3dd-6913-428b-9412-0d3947f1e166 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:24.447614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4f6aa3dd-6913-428b-9412-0d3947f1e166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:24.505719 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1282/2581] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:24.514065 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3fc13-75b0-4931-808b-b1caf676250e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:24.514394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3fc13-75b0-4931-808b-b1caf676250e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:24.514676 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3fc13-75b0-4931-808b-b1caf676250e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:24.532394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-13e3fc13-75b0-4931-808b-b1caf676250e None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:24.533608 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-13e3fc13-75b0-4931-808b-b1caf676250e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:24.541546 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1300/2582] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:24.546086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6aa3dd-6913-428b-9412-0d3947f1e166 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:24.556375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6aa3dd-6913-428b-9412-0d3947f1e166 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:24.562354 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f2972-fc2c-46d8-8049-b54ad7e460cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:24.563029 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f2972-fc2c-46d8-8049-b54ad7e460cf admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:24.563315 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f2972-fc2c-46d8-8049-b54ad7e460cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:24.563594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f2972-fc2c-46d8-8049-b54ad7e460cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:24.564316 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f2972-fc2c-46d8-8049-b54ad7e460cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:39:24.564622 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f2972-fc2c-46d8-8049-b54ad7e460cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:24.566388 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f2972-fc2c-46d8-8049-b54ad7e460cf admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:24.567881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:24.580722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1283/2583] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:39:24.584841 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3fc13-75b0-4931-808b-b1caf676250e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:24.594076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3fc13-75b0-4931-808b-b1caf676250e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:24.598692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d1901-9c7f-4065-a3a2-3f98985a9e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:24.599327 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d1901-9c7f-4065-a3a2-3f98985a9e3a admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:24.599609 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d1901-9c7f-4065-a3a2-3f98985a9e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:24.599887 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d1901-9c7f-4065-a3a2-3f98985a9e3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:24.600568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564d1901-9c7f-4065-a3a2-3f98985a9e3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:24.602508 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564d1901-9c7f-4065-a3a2-3f98985a9e3a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:24.615173 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:24.632343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1301/2584] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:39:24 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:39:24.636800 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f2972-fc2c-46d8-8049-b54ad7e460cf admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:24.646643 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f2972-fc2c-46d8-8049-b54ad7e460cf admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:24.702892 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6a435-40fb-4dbf-a5d5-2dfac968ab65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:24.703556 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6a435-40fb-4dbf-a5d5-2dfac968ab65 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:24.703846 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6a435-40fb-4dbf-a5d5-2dfac968ab65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:24.704129 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6a435-40fb-4dbf-a5d5-2dfac968ab65 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:24.704813 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6a435-40fb-4dbf-a5d5-2dfac968ab65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:24.707062 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6a435-40fb-4dbf-a5d5-2dfac968ab65 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:24.726864 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1284/2585] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:39:24 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:39:24.731668 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d1901-9c7f-4065-a3a2-3f98985a9e3a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:24.741791 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d1901-9c7f-4065-a3a2-3f98985a9e3a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:24.746374 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a712fe3-0f18-4a4e-91a2-6b86d476ba76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:24.747007 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a712fe3-0f18-4a4e-91a2-6b86d476ba76 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:24.747290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a712fe3-0f18-4a4e-91a2-6b86d476ba76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:24.747567 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a712fe3-0f18-4a4e-91a2-6b86d476ba76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:24.748280 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a712fe3-0f18-4a4e-91a2-6b86d476ba76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:24.749909 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a712fe3-0f18-4a4e-91a2-6b86d476ba76 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:24.760081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1302/2586] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:39:24.763963 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6a435-40fb-4dbf-a5d5-2dfac968ab65 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:24.774480 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6a435-40fb-4dbf-a5d5-2dfac968ab65 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:24.780452 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d5f33-02b8-4b42-a334-5ab01a78c264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:24.781141 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d5f33-02b8-4b42-a334-5ab01a78c264 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:24.781428 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d5f33-02b8-4b42-a334-5ab01a78c264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:24.781743 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d5f33-02b8-4b42-a334-5ab01a78c264 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:24.782478 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d5f33-02b8-4b42-a334-5ab01a78c264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:24.784087 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d5f33-02b8-4b42-a334-5ab01a78c264 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:24.794274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1285/2587] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:39:24.798527 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a712fe3-0f18-4a4e-91a2-6b86d476ba76 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:24.808973 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a712fe3-0f18-4a4e-91a2-6b86d476ba76 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:24.813772 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecdda77-99b9-48de-8ee7-59b36146ff47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:24.814658 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdda77-99b9-48de-8ee7-59b36146ff47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:24.814940 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdda77-99b9-48de-8ee7-59b36146ff47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:24.815218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdda77-99b9-48de-8ee7-59b36146ff47 admin admin] PATH_INFO: `/v3/projects/b364923b1c0a48bbbfeacb687a159318/users/d974131e371f44f98715d64ef999fd81/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:24.829703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecdda77-99b9-48de-8ee7-59b36146ff47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b364923b1c0a48bbbfeacb687a159318, user_id=d974131e371f44f98715d64ef999fd81, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:24.832620 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecdda77-99b9-48de-8ee7-59b36146ff47 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:24.845790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1303/2588] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:39:24 2022] PUT /identity/v3/projects/b364923b1c0a48bbbfeacb687a159318/users/d974131e371f44f98715d64ef999fd81/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:39:24.856424 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948eb56-5ab5-43df-9910-b38ab67ca5c2 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:24.856720 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948eb56-5ab5-43df-9910-b38ab67ca5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:24.857005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948eb56-5ab5-43df-9910-b38ab67ca5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:24.883369 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f948eb56-5ab5-43df-9910-b38ab67ca5c2 None None] MFA Rules not processed for user `d974131e371f44f98715d64ef999fd81`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:24.884626 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f948eb56-5ab5-43df-9910-b38ab67ca5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:24.947841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1286/2589] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:24.953181 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecdda77-99b9-48de-8ee7-59b36146ff47 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:25.021243 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecdda77-99b9-48de-8ee7-59b36146ff47 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:25.029829 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b175772-c7f4-46b1-90a7-64ce81c01b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:25.030516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b175772-c7f4-46b1-90a7-64ce81c01b13 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:25.030805 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b175772-c7f4-46b1-90a7-64ce81c01b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:25.031085 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b175772-c7f4-46b1-90a7-64ce81c01b13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:25.031939 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b175772-c7f4-46b1-90a7-64ce81c01b13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:25.034179 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b175772-c7f4-46b1-90a7-64ce81c01b13 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:25.065358 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1304/2590] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:39:24 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:39:25.066623 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948eb56-5ab5-43df-9910-b38ab67ca5c2 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:25.079088 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948eb56-5ab5-43df-9910-b38ab67ca5c2 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:25.139443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42980a15-63b4-4036-b3f8-ae0ea2d79d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:25.140149 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42980a15-63b4-4036-b3f8-ae0ea2d79d5b admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:25.140436 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42980a15-63b4-4036-b3f8-ae0ea2d79d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:25.140717 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42980a15-63b4-4036-b3f8-ae0ea2d79d5b admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:25.141437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42980a15-63b4-4036-b3f8-ae0ea2d79d5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:25.143441 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42980a15-63b4-4036-b3f8-ae0ea2d79d5b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:25.150960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:25.169893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1287/2591] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:39:25 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:39:25.171089 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b175772-c7f4-46b1-90a7-64ce81c01b13 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:25.181287 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b175772-c7f4-46b1-90a7-64ce81c01b13 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:25.186138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311f4a7-777f-4689-be56-4a3a5ceba763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:25.186781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311f4a7-777f-4689-be56-4a3a5ceba763 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:25.187066 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311f4a7-777f-4689-be56-4a3a5ceba763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:25.187345 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311f4a7-777f-4689-be56-4a3a5ceba763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:25.188058 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311f4a7-777f-4689-be56-4a3a5ceba763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:25.189684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311f4a7-777f-4689-be56-4a3a5ceba763 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:25.200800 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1305/2592] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:39:25.204884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42980a15-63b4-4036-b3f8-ae0ea2d79d5b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:25.215507 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42980a15-63b4-4036-b3f8-ae0ea2d79d5b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:25.220323 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f717afe-6bfa-4f7d-9a41-0fd73230dae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:25.221216 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f717afe-6bfa-4f7d-9a41-0fd73230dae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:25.221500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f717afe-6bfa-4f7d-9a41-0fd73230dae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:25.221810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f717afe-6bfa-4f7d-9a41-0fd73230dae0 admin admin] PATH_INFO: `/v3/projects/fbb74eb78c8c493e88206bbb2094363d/users/235298c6d24b4aefbb22c86f0a26382b/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:25.236862 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f717afe-6bfa-4f7d-9a41-0fd73230dae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbb74eb78c8c493e88206bbb2094363d, user_id=235298c6d24b4aefbb22c86f0a26382b, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:25.239575 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f717afe-6bfa-4f7d-9a41-0fd73230dae0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:25.253023 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1288/2593] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:39:25 2022] PUT /identity/v3/projects/fbb74eb78c8c493e88206bbb2094363d/users/235298c6d24b4aefbb22c86f0a26382b/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:39:25.256987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311f4a7-777f-4689-be56-4a3a5ceba763 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:25.267095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311f4a7-777f-4689-be56-4a3a5ceba763 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:25.322754 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed659633-c1e6-44a7-8205-f4883c1066ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:25.323606 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed659633-c1e6-44a7-8205-f4883c1066ba admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:25.323901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed659633-c1e6-44a7-8205-f4883c1066ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:25.324186 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed659633-c1e6-44a7-8205-f4883c1066ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:25.324913 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed659633-c1e6-44a7-8205-f4883c1066ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:25.326556 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed659633-c1e6-44a7-8205-f4883c1066ba admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:25.336490 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1306/2594] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:39:25.341054 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f717afe-6bfa-4f7d-9a41-0fd73230dae0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:25.351155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f717afe-6bfa-4f7d-9a41-0fd73230dae0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:25.356811 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549f55b-f014-4e71-88de-f293f4bd98a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:25.357740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549f55b-f014-4e71-88de-f293f4bd98a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:25.358027 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549f55b-f014-4e71-88de-f293f4bd98a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:25.358306 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549f55b-f014-4e71-88de-f293f4bd98a4 admin admin] PATH_INFO: `/v3/projects/fbb74eb78c8c493e88206bbb2094363d/users/235298c6d24b4aefbb22c86f0a26382b/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:25.360297 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549f55b-f014-4e71-88de-f293f4bd98a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbb74eb78c8c493e88206bbb2094363d, user_id=235298c6d24b4aefbb22c86f0a26382b, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:25.362985 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549f55b-f014-4e71-88de-f293f4bd98a4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:25.378258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1289/2595] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:39:25 2022] PUT /identity/v3/projects/fbb74eb78c8c493e88206bbb2094363d/users/235298c6d24b4aefbb22c86f0a26382b/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:25.390012 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab69a09-8456-4eab-aa39-3f285192f1ec None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:25.390303 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab69a09-8456-4eab-aa39-3f285192f1ec None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:25.390611 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab69a09-8456-4eab-aa39-3f285192f1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:25.417918 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fab69a09-8456-4eab-aa39-3f285192f1ec None None] MFA Rules not processed for user `235298c6d24b4aefbb22c86f0a26382b`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:25.419135 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fab69a09-8456-4eab-aa39-3f285192f1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:25.431334 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:25.486000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1307/2596] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:25 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:25.497418 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549f55b-f014-4e71-88de-f293f4bd98a4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:25.499179 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8549f55b-f014-4e71-88de-f293f4bd98a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:25.561415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549f55b-f014-4e71-88de-f293f4bd98a4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:25.564986 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea4967-d87a-4e13-9dda-b82e000e761f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:25.565646 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea4967-d87a-4e13-9dda-b82e000e761f service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:25.565931 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea4967-d87a-4e13-9dda-b82e000e761f service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:25.566208 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea4967-d87a-4e13-9dda-b82e000e761f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:25.575188 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea4967-d87a-4e13-9dda-b82e000e761f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:25.576626 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea4967-d87a-4e13-9dda-b82e000e761f service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:25.586370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1290/2597] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:39:25 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:39:25.975850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab69a09-8456-4eab-aa39-3f285192f1ec None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:26.037271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab69a09-8456-4eab-aa39-3f285192f1ec None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:26.042502 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29f2b9-05b9-48b5-b6cf-d047e22fba1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:26.043354 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29f2b9-05b9-48b5-b6cf-d047e22fba1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:26.043633 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29f2b9-05b9-48b5-b6cf-d047e22fba1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:26.043906 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29f2b9-05b9-48b5-b6cf-d047e22fba1e admin admin] PATH_INFO: `/v3/users/d974131e371f44f98715d64ef999fd81` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:26.045087 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f29f2b9-05b9-48b5-b6cf-d047e22fba1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d974131e371f44f98715d64ef999fd81)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:26.047041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f29f2b9-05b9-48b5-b6cf-d047e22fba1e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:26.092057 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9f29f2b9-05b9-48b5-b6cf-d047e22fba1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd974131e371f44f98715d64ef999fd81'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:26.103567 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9f29f2b9-05b9-48b5-b6cf-d047e22fba1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd974131e371f44f98715d64ef999fd81'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:26.104837 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9f29f2b9-05b9-48b5-b6cf-d047e22fba1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd974131e371f44f98715d64ef999fd81'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:26.111665 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9f29f2b9-05b9-48b5-b6cf-d047e22fba1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd974131e371f44f98715d64ef999fd81'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:26.112294 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9f29f2b9-05b9-48b5-b6cf-d047e22fba1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd974131e371f44f98715d64ef999fd81'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:26.125412 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1308/2598] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:39:25 2022] DELETE /identity/v3/users/d974131e371f44f98715d64ef999fd81 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:26.129916 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6eb6c8-044b-4a3f-8bc8-df137b3994a6 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:26.130403 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6eb6c8-044b-4a3f-8bc8-df137b3994a6 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:26.130508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6eb6c8-044b-4a3f-8bc8-df137b3994a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:26.151322 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-8f6eb6c8-044b-4a3f-8bc8-df137b3994a6 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:26.152813 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8f6eb6c8-044b-4a3f-8bc8-df137b3994a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:26.211268 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1291/2599] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:26.221931 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29f2b9-05b9-48b5-b6cf-d047e22fba1e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:26.223356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9f29f2b9-05b9-48b5-b6cf-d047e22fba1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:26.257038 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:26.281654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29f2b9-05b9-48b5-b6cf-d047e22fba1e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:26.285148 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab66776-edb9-4ada-83fd-d29ca3926d76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:26.285805 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab66776-edb9-4ada-83fd-d29ca3926d76 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:26.286091 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab66776-edb9-4ada-83fd-d29ca3926d76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:26.286368 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab66776-edb9-4ada-83fd-d29ca3926d76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:26.295779 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab66776-edb9-4ada-83fd-d29ca3926d76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:26.297199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab66776-edb9-4ada-83fd-d29ca3926d76 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:26.306414 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1309/2600] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:39:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:39:26.575585 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6eb6c8-044b-4a3f-8bc8-df137b3994a6 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:26.577012 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8f6eb6c8-044b-4a3f-8bc8-df137b3994a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:26.590527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6eb6c8-044b-4a3f-8bc8-df137b3994a6 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:26.595263 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8185dc5-2d68-4fe0-b514-8ee4b51663ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:26.596124 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8185dc5-2d68-4fe0-b514-8ee4b51663ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:26.596409 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8185dc5-2d68-4fe0-b514-8ee4b51663ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:26.596687 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8185dc5-2d68-4fe0-b514-8ee4b51663ce admin admin] PATH_INFO: `/v3/projects/b364923b1c0a48bbbfeacb687a159318` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:26.597906 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8185dc5-2d68-4fe0-b514-8ee4b51663ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=b364923b1c0a48bbbfeacb687a159318)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:26.599821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8185dc5-2d68-4fe0-b514-8ee4b51663ce admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:26.648056 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e8185dc5-2d68-4fe0-b514-8ee4b51663ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b364923b1c0a48bbbfeacb687a159318', 'request_id': 'req-e8185dc5-2d68-4fe0-b514-8ee4b51663ce'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:26.649914 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=4,keystone:SELECT=92,keystone:DELETE=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:26.656714 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e8185dc5-2d68-4fe0-b514-8ee4b51663ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b364923b1c0a48bbbfeacb687a159318', 'request_id': 'req-e8185dc5-2d68-4fe0-b514-8ee4b51663ce'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:26.662753 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e8185dc5-2d68-4fe0-b514-8ee4b51663ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b364923b1c0a48bbbfeacb687a159318', 'request_id': 'req-e8185dc5-2d68-4fe0-b514-8ee4b51663ce'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:26.675148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e8185dc5-2d68-4fe0-b514-8ee4b51663ce admin admin] The token cache is being invalidate because project b364923b1c0a48bbbfeacb687a159318 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:39:26.675434 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e8185dc5-2d68-4fe0-b514-8ee4b51663ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:26.676065 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e8185dc5-2d68-4fe0-b514-8ee4b51663ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:26.677667 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1292/2601] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:39:26 2022] DELETE /identity/v3/projects/b364923b1c0a48bbbfeacb687a159318 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:26.681759 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab66776-edb9-4ada-83fd-d29ca3926d76 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:26.691368 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab66776-edb9-4ada-83fd-d29ca3926d76 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:26.746686 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237190ca-9b85-4b2b-b3a2-c296bba6f0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:26.747548 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237190ca-9b85-4b2b-b3a2-c296bba6f0a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:26.747835 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237190ca-9b85-4b2b-b3a2-c296bba6f0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:26.748118 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237190ca-9b85-4b2b-b3a2-c296bba6f0a0 admin admin] PATH_INFO: `/v3/users/235298c6d24b4aefbb22c86f0a26382b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:26.749300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237190ca-9b85-4b2b-b3a2-c296bba6f0a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=235298c6d24b4aefbb22c86f0a26382b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:26.751261 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237190ca-9b85-4b2b-b3a2-c296bba6f0a0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:26.795933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-237190ca-9b85-4b2b-b3a2-c296bba6f0a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '235298c6d24b4aefbb22c86f0a26382b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:26.807375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-237190ca-9b85-4b2b-b3a2-c296bba6f0a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '235298c6d24b4aefbb22c86f0a26382b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:26.808036 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-237190ca-9b85-4b2b-b3a2-c296bba6f0a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '235298c6d24b4aefbb22c86f0a26382b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:26.815032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-237190ca-9b85-4b2b-b3a2-c296bba6f0a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '235298c6d24b4aefbb22c86f0a26382b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:26.815687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-237190ca-9b85-4b2b-b3a2-c296bba6f0a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '235298c6d24b4aefbb22c86f0a26382b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:26.828606 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1310/2602] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:39:26 2022] DELETE /identity/v3/users/235298c6d24b4aefbb22c86f0a26382b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:27.100994 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8185dc5-2d68-4fe0-b514-8ee4b51663ce admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:27.102686 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e8185dc5-2d68-4fe0-b514-8ee4b51663ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:27.117554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8185dc5-2d68-4fe0-b514-8ee4b51663ce admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:27.174225 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ac27c-e45d-45df-a068-183d2a8efcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:27.175076 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ac27c-e45d-45df-a068-183d2a8efcc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:27.175362 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ac27c-e45d-45df-a068-183d2a8efcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:27.175636 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ac27c-e45d-45df-a068-183d2a8efcc7 admin admin] PATH_INFO: `/v3/projects/fbb74eb78c8c493e88206bbb2094363d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:27.176794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ac27c-e45d-45df-a068-183d2a8efcc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbb74eb78c8c493e88206bbb2094363d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:27.178719 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ac27c-e45d-45df-a068-183d2a8efcc7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:27.224387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bf2ac27c-e45d-45df-a068-183d2a8efcc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbb74eb78c8c493e88206bbb2094363d', 'request_id': 'req-bf2ac27c-e45d-45df-a068-183d2a8efcc7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:27.230454 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bf2ac27c-e45d-45df-a068-183d2a8efcc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbb74eb78c8c493e88206bbb2094363d', 'request_id': 'req-bf2ac27c-e45d-45df-a068-183d2a8efcc7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:27.235871 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bf2ac27c-e45d-45df-a068-183d2a8efcc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbb74eb78c8c493e88206bbb2094363d', 'request_id': 'req-bf2ac27c-e45d-45df-a068-183d2a8efcc7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:27.249238 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bf2ac27c-e45d-45df-a068-183d2a8efcc7 admin admin] The token cache is being invalidate because project fbb74eb78c8c493e88206bbb2094363d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:39:27.249551 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bf2ac27c-e45d-45df-a068-183d2a8efcc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:27.250213 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bf2ac27c-e45d-45df-a068-183d2a8efcc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:27.251750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1293/2603] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:39:27 2022] DELETE /identity/v3/projects/fbb74eb78c8c493e88206bbb2094363d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:27.256600 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73963b-51f9-47ce-9cc0-cfdd2d41d9fb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:27.257070 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73963b-51f9-47ce-9cc0-cfdd2d41d9fb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:27.257181 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73963b-51f9-47ce-9cc0-cfdd2d41d9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:27.275517 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-bb73963b-51f9-47ce-9cc0-cfdd2d41d9fb None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:27.276994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bb73963b-51f9-47ce-9cc0-cfdd2d41d9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:27.330041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:27.344121 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1311/2604] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:27.351810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3ba0f-06fe-42d8-b0cc-405bbb9d7e3e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:27.352085 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3ba0f-06fe-42d8-b0cc-405bbb9d7e3e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:27.352370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3ba0f-06fe-42d8-b0cc-405bbb9d7e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:27.371361 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-03e3ba0f-06fe-42d8-b0cc-405bbb9d7e3e None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:27.372568 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-03e3ba0f-06fe-42d8-b0cc-405bbb9d7e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:27.380499 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1294/2605] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:27.384583 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73963b-51f9-47ce-9cc0-cfdd2d41d9fb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:27.394541 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73963b-51f9-47ce-9cc0-cfdd2d41d9fb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:27.399272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc7baf-9747-4530-95dc-cd80862d6cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:27.399926 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc7baf-9747-4530-95dc-cd80862d6cfc admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:27.400216 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc7baf-9747-4530-95dc-cd80862d6cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:27.400955 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc7baf-9747-4530-95dc-cd80862d6cfc admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:27.401735 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc7baf-9747-4530-95dc-cd80862d6cfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:39:27.402068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc7baf-9747-4530-95dc-cd80862d6cfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:27.405456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc7baf-9747-4530-95dc-cd80862d6cfc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:27.422065 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1312/2606] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:39:27.426495 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3ba0f-06fe-42d8-b0cc-405bbb9d7e3e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:27.439699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3ba0f-06fe-42d8-b0cc-405bbb9d7e3e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:27.444733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e2f6b-23a5-4680-8c28-7d763e7c455b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:27.445408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e2f6b-23a5-4680-8c28-7d763e7c455b admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:27.445759 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e2f6b-23a5-4680-8c28-7d763e7c455b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:27.446041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e2f6b-23a5-4680-8c28-7d763e7c455b admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:27.446736 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e2f6b-23a5-4680-8c28-7d763e7c455b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:27.448745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e2f6b-23a5-4680-8c28-7d763e7c455b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:27.479228 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1295/2607] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:39:27 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:39:27.484852 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc7baf-9747-4530-95dc-cd80862d6cfc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:27.496638 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc7baf-9747-4530-95dc-cd80862d6cfc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:27.559547 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d619076-cb6e-482e-bfd9-64c873201477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:27.560807 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d619076-cb6e-482e-bfd9-64c873201477 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:27.560807 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d619076-cb6e-482e-bfd9-64c873201477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:27.560807 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d619076-cb6e-482e-bfd9-64c873201477 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:27.561444 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d619076-cb6e-482e-bfd9-64c873201477 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:27.564003 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d619076-cb6e-482e-bfd9-64c873201477 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:27.586745 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1313/2608] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:39:27 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:39:27.590945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e2f6b-23a5-4680-8c28-7d763e7c455b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:27.601967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e2f6b-23a5-4680-8c28-7d763e7c455b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:27.608400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d60d5-6f33-4ca9-ac63-cfd0423f3721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:27.609075 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d60d5-6f33-4ca9-ac63-cfd0423f3721 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:27.609362 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d60d5-6f33-4ca9-ac63-cfd0423f3721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:27.609677 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d60d5-6f33-4ca9-ac63-cfd0423f3721 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:27.610420 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d60d5-6f33-4ca9-ac63-cfd0423f3721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:27.612053 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d60d5-6f33-4ca9-ac63-cfd0423f3721 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:27.624702 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1296/2609] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:39:27.630696 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d619076-cb6e-482e-bfd9-64c873201477 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:27.643630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d619076-cb6e-482e-bfd9-64c873201477 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:27.648602 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461e06e-9fa1-4568-8995-db7262c3d713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:27.649277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461e06e-9fa1-4568-8995-db7262c3d713 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:27.649566 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461e06e-9fa1-4568-8995-db7262c3d713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:27.649900 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461e06e-9fa1-4568-8995-db7262c3d713 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:27.651404 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461e06e-9fa1-4568-8995-db7262c3d713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:27.653191 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461e06e-9fa1-4568-8995-db7262c3d713 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:27.663742 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1314/2610] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:39:27.669603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d60d5-6f33-4ca9-ac63-cfd0423f3721 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:27.680917 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d60d5-6f33-4ca9-ac63-cfd0423f3721 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:27.686103 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599526bc-c9f3-416d-9a97-05d9c61fab9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:27.686991 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599526bc-c9f3-416d-9a97-05d9c61fab9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:27.687276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599526bc-c9f3-416d-9a97-05d9c61fab9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:27.687554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599526bc-c9f3-416d-9a97-05d9c61fab9c admin admin] PATH_INFO: `/v3/projects/37b441e530a947009e2c308b6f73fb3d/users/41e3560760634bee850ab2b8c44c2b9d/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:27.704020 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599526bc-c9f3-416d-9a97-05d9c61fab9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=37b441e530a947009e2c308b6f73fb3d, user_id=41e3560760634bee850ab2b8c44c2b9d, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:27.707051 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599526bc-c9f3-416d-9a97-05d9c61fab9c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:27.722414 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1297/2611] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:39:27 2022] PUT /identity/v3/projects/37b441e530a947009e2c308b6f73fb3d/users/41e3560760634bee850ab2b8c44c2b9d/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:27.736068 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e61908-c986-49a9-a7f1-d3a199c96f08 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:27.736370 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e61908-c986-49a9-a7f1-d3a199c96f08 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:27.736660 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e61908-c986-49a9-a7f1-d3a199c96f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:27.771727 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-98e61908-c986-49a9-a7f1-d3a199c96f08 None None] MFA Rules not processed for user `41e3560760634bee850ab2b8c44c2b9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:27.773132 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-98e61908-c986-49a9-a7f1-d3a199c96f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:27.855091 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1315/2612] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:27.861360 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599526bc-c9f3-416d-9a97-05d9c61fab9c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:27.873797 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:27.940337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599526bc-c9f3-416d-9a97-05d9c61fab9c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:27.946764 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc572cc4-5914-438c-bde6-319a1360e071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:27.947574 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc572cc4-5914-438c-bde6-319a1360e071 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:27.947860 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc572cc4-5914-438c-bde6-319a1360e071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:27.948137 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc572cc4-5914-438c-bde6-319a1360e071 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:27.948856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc572cc4-5914-438c-bde6-319a1360e071 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:27.950822 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc572cc4-5914-438c-bde6-319a1360e071 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:27.980926 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1298/2613] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:39:27 2022] POST /identity/v3/projects => generated 363 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:39:27.985367 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e61908-c986-49a9-a7f1-d3a199c96f08 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:27.997622 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e61908-c986-49a9-a7f1-d3a199c96f08 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:28.045989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:28.060211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e6e67-c3a8-4814-9888-38c3dfbd226a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:28.060891 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e6e67-c3a8-4814-9888-38c3dfbd226a admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:28.061183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e6e67-c3a8-4814-9888-38c3dfbd226a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:28.061461 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e6e67-c3a8-4814-9888-38c3dfbd226a admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:28.062205 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e6e67-c3a8-4814-9888-38c3dfbd226a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:28.064148 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e6e67-c3a8-4814-9888-38c3dfbd226a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:28.086252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1316/2614] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:39:27 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:39:28.091118 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc572cc4-5914-438c-bde6-319a1360e071 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:28.104384 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc572cc4-5914-438c-bde6-319a1360e071 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:28.109335 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbfc1c-d8c5-45ae-86ec-46e66d4c34d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:28.109988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbfc1c-d8c5-45ae-86ec-46e66d4c34d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:28.110279 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbfc1c-d8c5-45ae-86ec-46e66d4c34d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:28.110560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbfc1c-d8c5-45ae-86ec-46e66d4c34d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:28.111288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afbfc1c-d8c5-45ae-86ec-46e66d4c34d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:28.112908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afbfc1c-d8c5-45ae-86ec-46e66d4c34d6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:28.127079 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1299/2615] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:39:28.131840 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e6e67-c3a8-4814-9888-38c3dfbd226a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:28.146456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e6e67-c3a8-4814-9888-38c3dfbd226a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:28.152899 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055561e-4760-4581-8aa3-680b5cef98a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:28.153844 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055561e-4760-4581-8aa3-680b5cef98a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:28.154132 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055561e-4760-4581-8aa3-680b5cef98a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:28.154408 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055561e-4760-4581-8aa3-680b5cef98a5 admin admin] PATH_INFO: `/v3/projects/cb0bd2806c374f07bc94d09a5f0146a2/users/1a805b3e9fcc4e5fb021435e9fa3fcd3/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:28.172539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055561e-4760-4581-8aa3-680b5cef98a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb0bd2806c374f07bc94d09a5f0146a2, user_id=1a805b3e9fcc4e5fb021435e9fa3fcd3, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:28.175383 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055561e-4760-4581-8aa3-680b5cef98a5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:28.189252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1317/2616] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:39:28 2022] PUT /identity/v3/projects/cb0bd2806c374f07bc94d09a5f0146a2/users/1a805b3e9fcc4e5fb021435e9fa3fcd3/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:28.193653 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbfc1c-d8c5-45ae-86ec-46e66d4c34d6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:28.204902 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbfc1c-d8c5-45ae-86ec-46e66d4c34d6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:28.261811 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152e4b1-7cdf-4980-a945-10f0982eaded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:28.262472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152e4b1-7cdf-4980-a945-10f0982eaded admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:28.262761 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152e4b1-7cdf-4980-a945-10f0982eaded admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:28.263042 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152e4b1-7cdf-4980-a945-10f0982eaded admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:28.263763 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152e4b1-7cdf-4980-a945-10f0982eaded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:28.265447 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152e4b1-7cdf-4980-a945-10f0982eaded admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:28.276154 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1300/2617] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:39:28.280415 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055561e-4760-4581-8aa3-680b5cef98a5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:28.291339 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055561e-4760-4581-8aa3-680b5cef98a5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:28.295983 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55092cb9-97e9-4e52-b86a-37e7d53f7ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:28.296861 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55092cb9-97e9-4e52-b86a-37e7d53f7ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:28.297147 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55092cb9-97e9-4e52-b86a-37e7d53f7ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:28.297426 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55092cb9-97e9-4e52-b86a-37e7d53f7ac8 admin admin] PATH_INFO: `/v3/projects/cb0bd2806c374f07bc94d09a5f0146a2/users/1a805b3e9fcc4e5fb021435e9fa3fcd3/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:28.299438 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55092cb9-97e9-4e52-b86a-37e7d53f7ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb0bd2806c374f07bc94d09a5f0146a2, user_id=1a805b3e9fcc4e5fb021435e9fa3fcd3, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:28.302179 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55092cb9-97e9-4e52-b86a-37e7d53f7ac8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:28.312360 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1318/2618] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:39:28 2022] PUT /identity/v3/projects/cb0bd2806c374f07bc94d09a5f0146a2/users/1a805b3e9fcc4e5fb021435e9fa3fcd3/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:28.323110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b60f8-0347-4dcd-9dc6-ee5886824f75 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:28.323401 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b60f8-0347-4dcd-9dc6-ee5886824f75 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:28.323700 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b60f8-0347-4dcd-9dc6-ee5886824f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:28.351479 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-527b60f8-0347-4dcd-9dc6-ee5886824f75 None None] MFA Rules not processed for user `1a805b3e9fcc4e5fb021435e9fa3fcd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:28.352703 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-527b60f8-0347-4dcd-9dc6-ee5886824f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:28.420673 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1301/2619] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:28 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:28.431766 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55092cb9-97e9-4e52-b86a-37e7d53f7ac8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:28.433481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-55092cb9-97e9-4e52-b86a-37e7d53f7ac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:28.495083 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55092cb9-97e9-4e52-b86a-37e7d53f7ac8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:28.498821 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca8267-9d5b-402b-bcf7-a1832cb93bf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:28.499471 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca8267-9d5b-402b-bcf7-a1832cb93bf1 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:28.499758 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca8267-9d5b-402b-bcf7-a1832cb93bf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:28.500046 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca8267-9d5b-402b-bcf7-a1832cb93bf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:28.509533 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca8267-9d5b-402b-bcf7-a1832cb93bf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:28.511200 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca8267-9d5b-402b-bcf7-a1832cb93bf1 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:28.520752 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1319/2620] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:39:28 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:39:28.591523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b60f8-0347-4dcd-9dc6-ee5886824f75 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:28.596051 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:28.602740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b60f8-0347-4dcd-9dc6-ee5886824f75 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:28.607103 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423765bf-227a-4184-aa46-b7daa5ca390d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:28.607737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423765bf-227a-4184-aa46-b7daa5ca390d service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:28.608046 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423765bf-227a-4184-aa46-b7daa5ca390d service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:28.608327 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423765bf-227a-4184-aa46-b7daa5ca390d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:28.674737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423765bf-227a-4184-aa46-b7daa5ca390d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:28.676096 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423765bf-227a-4184-aa46-b7daa5ca390d service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:28.685373 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1302/2621] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:39:28 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:39:31.390891 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca8267-9d5b-402b-bcf7-a1832cb93bf1 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:31.454980 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca8267-9d5b-402b-bcf7-a1832cb93bf1 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:31.459550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d94dc-6ef0-4147-be5e-8523fe63c740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:31.460409 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d94dc-6ef0-4147-be5e-8523fe63c740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:31.460694 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d94dc-6ef0-4147-be5e-8523fe63c740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:31.460975 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d94dc-6ef0-4147-be5e-8523fe63c740 admin admin] PATH_INFO: `/v3/users/41e3560760634bee850ab2b8c44c2b9d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:31.463113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d94dc-6ef0-4147-be5e-8523fe63c740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41e3560760634bee850ab2b8c44c2b9d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:31.465976 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d94dc-6ef0-4147-be5e-8523fe63c740 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:31.514741 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:31.516359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d72d94dc-6ef0-4147-be5e-8523fe63c740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41e3560760634bee850ab2b8c44c2b9d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:31.530883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d72d94dc-6ef0-4147-be5e-8523fe63c740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41e3560760634bee850ab2b8c44c2b9d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:31.531483 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d72d94dc-6ef0-4147-be5e-8523fe63c740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41e3560760634bee850ab2b8c44c2b9d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:31.538391 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d72d94dc-6ef0-4147-be5e-8523fe63c740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41e3560760634bee850ab2b8c44c2b9d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:31.539015 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d72d94dc-6ef0-4147-be5e-8523fe63c740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41e3560760634bee850ab2b8c44c2b9d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:31.553640 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1320/2622] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:39:31 2022] DELETE /identity/v3/users/41e3560760634bee850ab2b8c44c2b9d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:31.558667 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a0a7c-d584-4857-a951-9aa444a884b3 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:31.559134 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a0a7c-d584-4857-a951-9aa444a884b3 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:31.559272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a0a7c-d584-4857-a951-9aa444a884b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:31.576073 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-bf9a0a7c-d584-4857-a951-9aa444a884b3 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:31.577275 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bf9a0a7c-d584-4857-a951-9aa444a884b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:31.637748 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1303/2623] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:31.647317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d94dc-6ef0-4147-be5e-8523fe63c740 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:31.648718 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d72d94dc-6ef0-4147-be5e-8523fe63c740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:31.704988 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d94dc-6ef0-4147-be5e-8523fe63c740 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:31.708546 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b55f992-3ca8-484b-96e1-897457045ead service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:31.709178 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55f992-3ca8-484b-96e1-897457045ead service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:31.709462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55f992-3ca8-484b-96e1-897457045ead service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:31.709766 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55f992-3ca8-484b-96e1-897457045ead service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:31.719088 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b55f992-3ca8-484b-96e1-897457045ead service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:31.720520 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b55f992-3ca8-484b-96e1-897457045ead service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:31.730359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1321/2624] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:39:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:39:32.002200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a0a7c-d584-4857-a951-9aa444a884b3 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:32.003616 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bf9a0a7c-d584-4857-a951-9aa444a884b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:32.017662 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a0a7c-d584-4857-a951-9aa444a884b3 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:32.023853 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8a0cb-3111-4e55-917d-214795a14d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:32.025397 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8a0cb-3111-4e55-917d-214795a14d6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:32.025732 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8a0cb-3111-4e55-917d-214795a14d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:32.026015 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8a0cb-3111-4e55-917d-214795a14d6c admin admin] PATH_INFO: `/v3/projects/37b441e530a947009e2c308b6f73fb3d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:32.027244 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f8a0cb-3111-4e55-917d-214795a14d6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=37b441e530a947009e2c308b6f73fb3d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:32.029196 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f8a0cb-3111-4e55-917d-214795a14d6c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:32.075733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-06f8a0cb-3111-4e55-917d-214795a14d6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37b441e530a947009e2c308b6f73fb3d', 'request_id': 'req-06f8a0cb-3111-4e55-917d-214795a14d6c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:32.082026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-06f8a0cb-3111-4e55-917d-214795a14d6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37b441e530a947009e2c308b6f73fb3d', 'request_id': 'req-06f8a0cb-3111-4e55-917d-214795a14d6c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:32.088006 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-06f8a0cb-3111-4e55-917d-214795a14d6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37b441e530a947009e2c308b6f73fb3d', 'request_id': 'req-06f8a0cb-3111-4e55-917d-214795a14d6c'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:32.100031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-06f8a0cb-3111-4e55-917d-214795a14d6c admin admin] The token cache is being invalidate because project 37b441e530a947009e2c308b6f73fb3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:39:32.100316 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-06f8a0cb-3111-4e55-917d-214795a14d6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:32.100946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-06f8a0cb-3111-4e55-917d-214795a14d6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:32.102588 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1304/2625] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:39:32 2022] DELETE /identity/v3/projects/37b441e530a947009e2c308b6f73fb3d => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:32.106630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b55f992-3ca8-484b-96e1-897457045ead service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:32.116744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b55f992-3ca8-484b-96e1-897457045ead service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:32.171662 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea82f87-315b-4fbb-b029-818c4d6fdfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:32.172669 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea82f87-315b-4fbb-b029-818c4d6fdfc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:32.172973 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea82f87-315b-4fbb-b029-818c4d6fdfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:32.173279 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea82f87-315b-4fbb-b029-818c4d6fdfc5 admin admin] PATH_INFO: `/v3/users/1a805b3e9fcc4e5fb021435e9fa3fcd3` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:32.174631 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea82f87-315b-4fbb-b029-818c4d6fdfc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a805b3e9fcc4e5fb021435e9fa3fcd3)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:32.176765 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea82f87-315b-4fbb-b029-818c4d6fdfc5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:32.222773 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9ea82f87-315b-4fbb-b029-818c4d6fdfc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a805b3e9fcc4e5fb021435e9fa3fcd3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:32.234500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9ea82f87-315b-4fbb-b029-818c4d6fdfc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a805b3e9fcc4e5fb021435e9fa3fcd3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:32.235176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9ea82f87-315b-4fbb-b029-818c4d6fdfc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a805b3e9fcc4e5fb021435e9fa3fcd3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:32.237136 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:32.243517 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9ea82f87-315b-4fbb-b029-818c4d6fdfc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a805b3e9fcc4e5fb021435e9fa3fcd3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:32.244691 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9ea82f87-315b-4fbb-b029-818c4d6fdfc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a805b3e9fcc4e5fb021435e9fa3fcd3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:32.258422 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1322/2626] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:39:32 2022] DELETE /identity/v3/users/1a805b3e9fcc4e5fb021435e9fa3fcd3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:32.522239 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8a0cb-3111-4e55-917d-214795a14d6c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:32.523791 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-06f8a0cb-3111-4e55-917d-214795a14d6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:32.540291 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8a0cb-3111-4e55-917d-214795a14d6c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:32.561363 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:32.599123 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f888fb-9670-4053-9ba8-691430f762ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:32.599973 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f888fb-9670-4053-9ba8-691430f762ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:32.600254 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f888fb-9670-4053-9ba8-691430f762ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:32.600529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f888fb-9670-4053-9ba8-691430f762ba admin admin] PATH_INFO: `/v3/projects/cb0bd2806c374f07bc94d09a5f0146a2` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:32.601838 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f888fb-9670-4053-9ba8-691430f762ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb0bd2806c374f07bc94d09a5f0146a2)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:32.604840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f888fb-9670-4053-9ba8-691430f762ba admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:32.650891 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f7f888fb-9670-4053-9ba8-691430f762ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb0bd2806c374f07bc94d09a5f0146a2', 'request_id': 'req-f7f888fb-9670-4053-9ba8-691430f762ba'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:32.656925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f7f888fb-9670-4053-9ba8-691430f762ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb0bd2806c374f07bc94d09a5f0146a2', 'request_id': 'req-f7f888fb-9670-4053-9ba8-691430f762ba'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:32.662471 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f7f888fb-9670-4053-9ba8-691430f762ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb0bd2806c374f07bc94d09a5f0146a2', 'request_id': 'req-f7f888fb-9670-4053-9ba8-691430f762ba'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:32.677872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f7f888fb-9670-4053-9ba8-691430f762ba admin admin] The token cache is being invalidate because project cb0bd2806c374f07bc94d09a5f0146a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:39:32.678162 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f7f888fb-9670-4053-9ba8-691430f762ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:32.678811 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-f7f888fb-9670-4053-9ba8-691430f762ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:32.680357 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1305/2627] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:39:32 2022] DELETE /identity/v3/projects/cb0bd2806c374f07bc94d09a5f0146a2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:32.685962 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a85d0-41e6-448d-ab76-c3d5dc600367 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:32.686427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a85d0-41e6-448d-ab76-c3d5dc600367 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:32.686635 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a85d0-41e6-448d-ab76-c3d5dc600367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:32.705004 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-350a85d0-41e6-448d-ab76-c3d5dc600367 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:32.706244 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-350a85d0-41e6-448d-ab76-c3d5dc600367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:32.765659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1323/2628] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:39:32.773808 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31295c-aa06-4138-a45f-47e8b75ab0a8 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:32.774145 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31295c-aa06-4138-a45f-47e8b75ab0a8 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:32.774437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31295c-aa06-4138-a45f-47e8b75ab0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:32.794166 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9c31295c-aa06-4138-a45f-47e8b75ab0a8 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:32.795493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9c31295c-aa06-4138-a45f-47e8b75ab0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:32.803607 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1306/2629] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:32.808541 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a85d0-41e6-448d-ab76-c3d5dc600367 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:32.818276 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a85d0-41e6-448d-ab76-c3d5dc600367 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:32.823151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff06368-4be2-4b5e-bb89-fab21eca3adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:32.823802 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff06368-4be2-4b5e-bb89-fab21eca3adc admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:32.824087 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff06368-4be2-4b5e-bb89-fab21eca3adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:32.824367 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff06368-4be2-4b5e-bb89-fab21eca3adc admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:32.825092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff06368-4be2-4b5e-bb89-fab21eca3adc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:39:32.825400 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff06368-4be2-4b5e-bb89-fab21eca3adc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:32.827030 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff06368-4be2-4b5e-bb89-fab21eca3adc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:32.839047 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1324/2630] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:39:32.843290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31295c-aa06-4138-a45f-47e8b75ab0a8 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:32.853969 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31295c-aa06-4138-a45f-47e8b75ab0a8 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:32.858696 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d8b9cf-1d1e-4083-94d6-685b7740d1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:32.859278 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8b9cf-1d1e-4083-94d6-685b7740d1fc admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:32.859563 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8b9cf-1d1e-4083-94d6-685b7740d1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:32.859837 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8b9cf-1d1e-4083-94d6-685b7740d1fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:32.861394 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d8b9cf-1d1e-4083-94d6-685b7740d1fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:32.863361 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d8b9cf-1d1e-4083-94d6-685b7740d1fc admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:32.889927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1307/2631] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:39:32 2022] POST /identity/v3/projects => generated 393 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:39:32.893849 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff06368-4be2-4b5e-bb89-fab21eca3adc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:32.904202 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff06368-4be2-4b5e-bb89-fab21eca3adc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:32.959462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ce7d8-3282-4033-8585-0859ce463b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:32.960102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ce7d8-3282-4033-8585-0859ce463b50 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:32.960386 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ce7d8-3282-4033-8585-0859ce463b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:32.960665 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ce7d8-3282-4033-8585-0859ce463b50 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:32.961354 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ce7d8-3282-4033-8585-0859ce463b50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:32.964228 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ce7d8-3282-4033-8585-0859ce463b50 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:32.986319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1325/2632] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:39:32 2022] POST /identity/v3/users => generated 297 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:39:32.987538 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d8b9cf-1d1e-4083-94d6-685b7740d1fc admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:32.999123 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d8b9cf-1d1e-4083-94d6-685b7740d1fc admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:33.003790 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101c9b0-386d-48e1-8cf6-c3020f6ef1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:33.004437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101c9b0-386d-48e1-8cf6-c3020f6ef1ce admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:33.004723 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101c9b0-386d-48e1-8cf6-c3020f6ef1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:33.005005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101c9b0-386d-48e1-8cf6-c3020f6ef1ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:33.005773 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101c9b0-386d-48e1-8cf6-c3020f6ef1ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:33.007373 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101c9b0-386d-48e1-8cf6-c3020f6ef1ce admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:33.018613 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1308/2633] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:39:33.022475 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ce7d8-3282-4033-8585-0859ce463b50 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:33.032941 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ce7d8-3282-4033-8585-0859ce463b50 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:33.037510 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d2773-e594-430f-9640-6cdba7444932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:33.038208 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d2773-e594-430f-9640-6cdba7444932 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:33.038496 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d2773-e594-430f-9640-6cdba7444932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:33.038765 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d2773-e594-430f-9640-6cdba7444932 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:33.039489 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d2773-e594-430f-9640-6cdba7444932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:33.041108 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d2773-e594-430f-9640-6cdba7444932 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:33.051161 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1326/2634] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:39:33.055207 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101c9b0-386d-48e1-8cf6-c3020f6ef1ce admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:33.066034 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101c9b0-386d-48e1-8cf6-c3020f6ef1ce admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:33.070704 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01debe-35cf-46c7-9781-1e9455d14d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:33.071624 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01debe-35cf-46c7-9781-1e9455d14d90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:33.071909 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01debe-35cf-46c7-9781-1e9455d14d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:33.072186 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01debe-35cf-46c7-9781-1e9455d14d90 admin admin] PATH_INFO: `/v3/projects/030d422f1b214075b6492d91f26272be/users/7f94410cdadb45978afaf26226368b73/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:33.087449 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db01debe-35cf-46c7-9781-1e9455d14d90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=030d422f1b214075b6492d91f26272be, user_id=7f94410cdadb45978afaf26226368b73, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:33.090158 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db01debe-35cf-46c7-9781-1e9455d14d90 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:33.103572 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1309/2635] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:39:33 2022] PUT /identity/v3/projects/030d422f1b214075b6492d91f26272be/users/7f94410cdadb45978afaf26226368b73/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:33.115161 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a43ae1-91c5-457f-ae02-339ad17caecb None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:33.115372 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a43ae1-91c5-457f-ae02-339ad17caecb None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:33.115655 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a43ae1-91c5-457f-ae02-339ad17caecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:33.142728 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-11a43ae1-91c5-457f-ae02-339ad17caecb None None] MFA Rules not processed for user `7f94410cdadb45978afaf26226368b73`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:33.143942 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-11a43ae1-91c5-457f-ae02-339ad17caecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:33.184696 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:33.210677 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1327/2636] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:33.215787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01debe-35cf-46c7-9781-1e9455d14d90 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:33.267002 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:33.280100 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01debe-35cf-46c7-9781-1e9455d14d90 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:33.285183 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b67da-7fdf-4bc0-aec6-23f361856d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:33.285866 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b67da-7fdf-4bc0-aec6-23f361856d65 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:33.286153 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b67da-7fdf-4bc0-aec6-23f361856d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:33.286498 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b67da-7fdf-4bc0-aec6-23f361856d65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:33.287196 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b67da-7fdf-4bc0-aec6-23f361856d65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:33.289121 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b67da-7fdf-4bc0-aec6-23f361856d65 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:33.318944 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1310/2637] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:39:33 2022] POST /identity/v3/projects => generated 389 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:39:33.320121 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a43ae1-91c5-457f-ae02-339ad17caecb None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:33.331717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a43ae1-91c5-457f-ae02-339ad17caecb None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:33.386914 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88bc453-dcc4-4a50-99aa-e4ab6708a6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:33.387508 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bc453-dcc4-4a50-99aa-e4ab6708a6a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:33.387796 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bc453-dcc4-4a50-99aa-e4ab6708a6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:33.388079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bc453-dcc4-4a50-99aa-e4ab6708a6a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:33.388774 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88bc453-dcc4-4a50-99aa-e4ab6708a6a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:33.390764 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88bc453-dcc4-4a50-99aa-e4ab6708a6a9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:33.411395 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1328/2638] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:39:33 2022] POST /identity/v3/users => generated 301 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:39:33.415949 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b67da-7fdf-4bc0-aec6-23f361856d65 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:33.426387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b67da-7fdf-4bc0-aec6-23f361856d65 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:33.431560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e8cf8-ba1e-4bbb-838f-cf8bb2c7e4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:33.432208 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e8cf8-ba1e-4bbb-838f-cf8bb2c7e4a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:33.432493 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e8cf8-ba1e-4bbb-838f-cf8bb2c7e4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:33.432773 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e8cf8-ba1e-4bbb-838f-cf8bb2c7e4a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:33.433498 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e8cf8-ba1e-4bbb-838f-cf8bb2c7e4a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:33.435202 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e8cf8-ba1e-4bbb-838f-cf8bb2c7e4a2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:33.445534 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1311/2639] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:39:33.450369 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88bc453-dcc4-4a50-99aa-e4ab6708a6a9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:33.460317 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88bc453-dcc4-4a50-99aa-e4ab6708a6a9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:33.465919 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb508e1d-06d0-40b9-8140-0bf618a48fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:33.466792 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb508e1d-06d0-40b9-8140-0bf618a48fc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:33.467081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb508e1d-06d0-40b9-8140-0bf618a48fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:33.467381 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb508e1d-06d0-40b9-8140-0bf618a48fc9 admin admin] PATH_INFO: `/v3/projects/85c187b0543847aaba7f9ddd131833fa/users/595e1f5dd37b4057ace9f6cb29ed5c71/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:33.482245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb508e1d-06d0-40b9-8140-0bf618a48fc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85c187b0543847aaba7f9ddd131833fa, user_id=595e1f5dd37b4057ace9f6cb29ed5c71, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:33.484922 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb508e1d-06d0-40b9-8140-0bf618a48fc9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:33.498310 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1329/2640] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:39:33 2022] PUT /identity/v3/projects/85c187b0543847aaba7f9ddd131833fa/users/595e1f5dd37b4057ace9f6cb29ed5c71/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:39:33.502401 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e8cf8-ba1e-4bbb-838f-cf8bb2c7e4a2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:33.513278 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e8cf8-ba1e-4bbb-838f-cf8bb2c7e4a2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:33.569282 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08720d6a-a969-4303-b63b-0fa2f050931d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:33.569981 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08720d6a-a969-4303-b63b-0fa2f050931d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:33.570271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08720d6a-a969-4303-b63b-0fa2f050931d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:33.570569 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08720d6a-a969-4303-b63b-0fa2f050931d admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:33.572205 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08720d6a-a969-4303-b63b-0fa2f050931d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:33.574053 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08720d6a-a969-4303-b63b-0fa2f050931d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:33.584056 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1312/2641] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:39:33.588723 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb508e1d-06d0-40b9-8140-0bf618a48fc9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:33.599010 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb508e1d-06d0-40b9-8140-0bf618a48fc9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:33.603870 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1044eb07-f44c-4e53-a7fd-6afd727ddc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:33.604779 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044eb07-f44c-4e53-a7fd-6afd727ddc8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:33.605069 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044eb07-f44c-4e53-a7fd-6afd727ddc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:33.605347 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044eb07-f44c-4e53-a7fd-6afd727ddc8b admin admin] PATH_INFO: `/v3/projects/85c187b0543847aaba7f9ddd131833fa/users/595e1f5dd37b4057ace9f6cb29ed5c71/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:33.607387 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1044eb07-f44c-4e53-a7fd-6afd727ddc8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=85c187b0543847aaba7f9ddd131833fa, user_id=595e1f5dd37b4057ace9f6cb29ed5c71, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:33.610168 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1044eb07-f44c-4e53-a7fd-6afd727ddc8b admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:33.620401 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1330/2642] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:39:33 2022] PUT /identity/v3/projects/85c187b0543847aaba7f9ddd131833fa/users/595e1f5dd37b4057ace9f6cb29ed5c71/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:39:33.630667 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0bf4c-d7b2-4c38-b51c-a104c14e65d1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:33.630965 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0bf4c-d7b2-4c38-b51c-a104c14e65d1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:33.631251 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0bf4c-d7b2-4c38-b51c-a104c14e65d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:33.658422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-37c0bf4c-d7b2-4c38-b51c-a104c14e65d1 None None] MFA Rules not processed for user `595e1f5dd37b4057ace9f6cb29ed5c71`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:33.659568 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-37c0bf4c-d7b2-4c38-b51c-a104c14e65d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:33.728383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1313/2643] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:33 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:33.738900 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1044eb07-f44c-4e53-a7fd-6afd727ddc8b admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:33.740686 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1044eb07-f44c-4e53-a7fd-6afd727ddc8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:33.797212 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1044eb07-f44c-4e53-a7fd-6afd727ddc8b admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:33.800958 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97cab65-bc70-4c31-b529-6045ecd66a93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:33.801632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97cab65-bc70-4c31-b529-6045ecd66a93 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:33.801927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97cab65-bc70-4c31-b529-6045ecd66a93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:33.802211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97cab65-bc70-4c31-b529-6045ecd66a93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:33.811881 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97cab65-bc70-4c31-b529-6045ecd66a93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:33.813323 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97cab65-bc70-4c31-b529-6045ecd66a93 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:33.823458 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1331/2644] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:39:33 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:39:33.869291 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0bf4c-d7b2-4c38-b51c-a104c14e65d1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:33.870722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-37c0bf4c-d7b2-4c38-b51c-a104c14e65d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:33.882772 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0bf4c-d7b2-4c38-b51c-a104c14e65d1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:33.886727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7815f47-e10f-42b2-82cc-969e0e417a71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:33.887364 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7815f47-e10f-42b2-82cc-969e0e417a71 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:33.887649 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7815f47-e10f-42b2-82cc-969e0e417a71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:33.887931 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7815f47-e10f-42b2-82cc-969e0e417a71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:33.932913 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:33.946251 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7815f47-e10f-42b2-82cc-969e0e417a71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:33.947700 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7815f47-e10f-42b2-82cc-969e0e417a71 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:33.956888 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1314/2645] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:39:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:39:36.184904 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97cab65-bc70-4c31-b529-6045ecd66a93 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:36.186598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e97cab65-bc70-4c31-b529-6045ecd66a93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:36.232356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:36.252775 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97cab65-bc70-4c31-b529-6045ecd66a93 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:36.256893 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a532a02-6c58-415e-8d17-155f077ad8d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:36.257530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a532a02-6c58-415e-8d17-155f077ad8d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:36.258879 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a532a02-6c58-415e-8d17-155f077ad8d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:36.259168 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a532a02-6c58-415e-8d17-155f077ad8d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:36.260955 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8a532a02-6c58-415e-8d17-155f077ad8d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:36.270387 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a532a02-6c58-415e-8d17-155f077ad8d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:36.271818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a532a02-6c58-415e-8d17-155f077ad8d1 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:36.280998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1332/2646] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:39:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:39:37.183304 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7815f47-e10f-42b2-82cc-969e0e417a71 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:37.245199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7815f47-e10f-42b2-82cc-969e0e417a71 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:37.249644 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574baf78-eb6e-4750-a213-96336353ea1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:37.250497 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574baf78-eb6e-4750-a213-96336353ea1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:37.250795 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574baf78-eb6e-4750-a213-96336353ea1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:37.251071 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574baf78-eb6e-4750-a213-96336353ea1a admin admin] PATH_INFO: `/v3/users/7f94410cdadb45978afaf26226368b73` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:37.252229 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574baf78-eb6e-4750-a213-96336353ea1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f94410cdadb45978afaf26226368b73)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:37.254197 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574baf78-eb6e-4750-a213-96336353ea1a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:37.298634 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-574baf78-eb6e-4750-a213-96336353ea1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f94410cdadb45978afaf26226368b73'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:37.299192 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-574baf78-eb6e-4750-a213-96336353ea1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f94410cdadb45978afaf26226368b73'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:37.311807 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-574baf78-eb6e-4750-a213-96336353ea1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f94410cdadb45978afaf26226368b73'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:37.318922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-574baf78-eb6e-4750-a213-96336353ea1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f94410cdadb45978afaf26226368b73'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:37.319590 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-574baf78-eb6e-4750-a213-96336353ea1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f94410cdadb45978afaf26226368b73'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:37.331842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1315/2647] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:39:37 2022] DELETE /identity/v3/users/7f94410cdadb45978afaf26226368b73 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:37.336626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd9e0d-745b-4bcf-98d2-18328d57beb9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:37.337113 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd9e0d-745b-4bcf-98d2-18328d57beb9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:37.337228 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd9e0d-745b-4bcf-98d2-18328d57beb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:37.356036 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2fdd9e0d-745b-4bcf-98d2-18328d57beb9 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:37.357238 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2fdd9e0d-745b-4bcf-98d2-18328d57beb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:37.419695 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1333/2648] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:37.429636 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574baf78-eb6e-4750-a213-96336353ea1a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:37.431087 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-574baf78-eb6e-4750-a213-96336353ea1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:37.491761 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574baf78-eb6e-4750-a213-96336353ea1a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:37.495553 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977d6c4-59a7-4bc1-80d2-a7b9f735b04f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:37.496206 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977d6c4-59a7-4bc1-80d2-a7b9f735b04f service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:37.496494 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977d6c4-59a7-4bc1-80d2-a7b9f735b04f service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:37.496778 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977d6c4-59a7-4bc1-80d2-a7b9f735b04f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:37.506552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3977d6c4-59a7-4bc1-80d2-a7b9f735b04f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:37.508786 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3977d6c4-59a7-4bc1-80d2-a7b9f735b04f service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:37.518389 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1316/2649] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:39:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:39:38.044975 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd9e0d-745b-4bcf-98d2-18328d57beb9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:38.046847 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2fdd9e0d-745b-4bcf-98d2-18328d57beb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:38.062309 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd9e0d-745b-4bcf-98d2-18328d57beb9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:38.067668 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be63563-4366-4b76-b015-bd8625993a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:38.068622 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be63563-4366-4b76-b015-bd8625993a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:38.069024 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be63563-4366-4b76-b015-bd8625993a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:38.069330 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be63563-4366-4b76-b015-bd8625993a83 admin admin] PATH_INFO: `/v3/projects/030d422f1b214075b6492d91f26272be` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:38.071195 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be63563-4366-4b76-b015-bd8625993a83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=030d422f1b214075b6492d91f26272be)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:38.073063 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be63563-4366-4b76-b015-bd8625993a83 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:38.120259 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6be63563-4366-4b76-b015-bd8625993a83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '030d422f1b214075b6492d91f26272be', 'request_id': 'req-6be63563-4366-4b76-b015-bd8625993a83'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:38.126295 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6be63563-4366-4b76-b015-bd8625993a83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '030d422f1b214075b6492d91f26272be', 'request_id': 'req-6be63563-4366-4b76-b015-bd8625993a83'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:38.131594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6be63563-4366-4b76-b015-bd8625993a83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '030d422f1b214075b6492d91f26272be', 'request_id': 'req-6be63563-4366-4b76-b015-bd8625993a83'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:38.143281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6be63563-4366-4b76-b015-bd8625993a83 admin admin] The token cache is being invalidate because project 030d422f1b214075b6492d91f26272be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:39:38.143571 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6be63563-4366-4b76-b015-bd8625993a83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:38.144185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-6be63563-4366-4b76-b015-bd8625993a83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:38.145726 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1334/2650] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:39:38 2022] DELETE /identity/v3/projects/030d422f1b214075b6492d91f26272be => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:38.150522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977d6c4-59a7-4bc1-80d2-a7b9f735b04f service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:38.159956 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977d6c4-59a7-4bc1-80d2-a7b9f735b04f service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:38.164256 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:38.219912 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cdfe2-5d68-451d-a247-c91d07a63921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:38.220781 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cdfe2-5d68-451d-a247-c91d07a63921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:38.221061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cdfe2-5d68-451d-a247-c91d07a63921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:38.221336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cdfe2-5d68-451d-a247-c91d07a63921 admin admin] PATH_INFO: `/v3/users/595e1f5dd37b4057ace9f6cb29ed5c71` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:38.222552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208cdfe2-5d68-451d-a247-c91d07a63921 admin admin] RBAC: Authorizing `identity:delete_user(user_id=595e1f5dd37b4057ace9f6cb29ed5c71)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:38.224461 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208cdfe2-5d68-451d-a247-c91d07a63921 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:38.272164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-208cdfe2-5d68-451d-a247-c91d07a63921 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '595e1f5dd37b4057ace9f6cb29ed5c71'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:38.273466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-208cdfe2-5d68-451d-a247-c91d07a63921 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '595e1f5dd37b4057ace9f6cb29ed5c71'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:38.285811 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-208cdfe2-5d68-451d-a247-c91d07a63921 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '595e1f5dd37b4057ace9f6cb29ed5c71'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:38.294777 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-208cdfe2-5d68-451d-a247-c91d07a63921 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '595e1f5dd37b4057ace9f6cb29ed5c71'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:38.295464 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-208cdfe2-5d68-451d-a247-c91d07a63921 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '595e1f5dd37b4057ace9f6cb29ed5c71'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:38.308500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1317/2651] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:39:38 2022] DELETE /identity/v3/users/595e1f5dd37b4057ace9f6cb29ed5c71 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:38.576335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be63563-4366-4b76-b015-bd8625993a83 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:38.578104 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6be63563-4366-4b76-b015-bd8625993a83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:38.592144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be63563-4366-4b76-b015-bd8625993a83 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:38.653289 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8d8da-33ae-4b72-99b4-3d13b8a4c71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:38.654213 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8d8da-33ae-4b72-99b4-3d13b8a4c71d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:38.654501 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8d8da-33ae-4b72-99b4-3d13b8a4c71d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:38.654790 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8d8da-33ae-4b72-99b4-3d13b8a4c71d admin admin] PATH_INFO: `/v3/projects/85c187b0543847aaba7f9ddd131833fa` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:38.656320 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8d8da-33ae-4b72-99b4-3d13b8a4c71d admin admin] RBAC: Authorizing `identity:delete_project(project_id=85c187b0543847aaba7f9ddd131833fa)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:38.658863 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8d8da-33ae-4b72-99b4-3d13b8a4c71d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:38.661396 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:38.709343 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c0d8d8da-33ae-4b72-99b4-3d13b8a4c71d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85c187b0543847aaba7f9ddd131833fa', 'request_id': 'req-c0d8d8da-33ae-4b72-99b4-3d13b8a4c71d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:38.715750 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c0d8d8da-33ae-4b72-99b4-3d13b8a4c71d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85c187b0543847aaba7f9ddd131833fa', 'request_id': 'req-c0d8d8da-33ae-4b72-99b4-3d13b8a4c71d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:38.721319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c0d8d8da-33ae-4b72-99b4-3d13b8a4c71d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85c187b0543847aaba7f9ddd131833fa', 'request_id': 'req-c0d8d8da-33ae-4b72-99b4-3d13b8a4c71d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:38.728072 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cdfe2-5d68-451d-a247-c91d07a63921 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:38.729653 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-208cdfe2-5d68-451d-a247-c91d07a63921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:38.735179 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c0d8d8da-33ae-4b72-99b4-3d13b8a4c71d admin admin] The token cache is being invalidate because project 85c187b0543847aaba7f9ddd131833fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:39:38.735472 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c0d8d8da-33ae-4b72-99b4-3d13b8a4c71d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:38.736109 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c0d8d8da-33ae-4b72-99b4-3d13b8a4c71d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:38.737844 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1335/2652] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:39:38 2022] DELETE /identity/v3/projects/85c187b0543847aaba7f9ddd131833fa => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:38.743321 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f457143-ec81-45c7-8bfa-c84d9e800e7f None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:38.743617 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f457143-ec81-45c7-8bfa-c84d9e800e7f None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:38.743897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f457143-ec81-45c7-8bfa-c84d9e800e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:38.762243 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6f457143-ec81-45c7-8bfa-c84d9e800e7f None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:38.763456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6f457143-ec81-45c7-8bfa-c84d9e800e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:38.790678 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cdfe2-5d68-451d-a247-c91d07a63921 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:38.823819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1336/2653] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:39:38.832500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc6fa3-3384-4946-ba4b-23b7b29185e9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:38.832669 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:38.832815 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc6fa3-3384-4946-ba4b-23b7b29185e9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:38.833079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc6fa3-3384-4946-ba4b-23b7b29185e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:38.843369 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4847ac8-5d31-44c8-878f-6358ec271dc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:38.844125 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4847ac8-5d31-44c8-878f-6358ec271dc9 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:38.844417 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4847ac8-5d31-44c8-878f-6358ec271dc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:38.844961 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4847ac8-5d31-44c8-878f-6358ec271dc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:38.846852 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e4847ac8-5d31-44c8-878f-6358ec271dc9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:38.852663 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-aedc6fa3-3384-4946-ba4b-23b7b29185e9 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:38.853905 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-aedc6fa3-3384-4946-ba4b-23b7b29185e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:38.861930 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1337/2654] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:39:38.866209 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc6fa3-3384-4946-ba4b-23b7b29185e9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:38.876426 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc6fa3-3384-4946-ba4b-23b7b29185e9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:38.881031 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192f9af-0625-4765-b6b7-c3f989bd050e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:38.881698 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192f9af-0625-4765-b6b7-c3f989bd050e admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:38.881989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192f9af-0625-4765-b6b7-c3f989bd050e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:38.882257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192f9af-0625-4765-b6b7-c3f989bd050e admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:38.882974 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192f9af-0625-4765-b6b7-c3f989bd050e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:39:38.883277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192f9af-0625-4765-b6b7-c3f989bd050e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:38.884867 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192f9af-0625-4765-b6b7-c3f989bd050e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:38.896266 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1338/2655] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:39:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:39:38.901206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192f9af-0625-4765-b6b7-c3f989bd050e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:38.910678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192f9af-0625-4765-b6b7-c3f989bd050e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:38.916348 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cafd60f-7a86-4f76-ba37-467561cdd809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:38.916986 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafd60f-7a86-4f76-ba37-467561cdd809 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:38.917285 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafd60f-7a86-4f76-ba37-467561cdd809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:38.917569 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafd60f-7a86-4f76-ba37-467561cdd809 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:38.918282 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cafd60f-7a86-4f76-ba37-467561cdd809 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:38.920281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cafd60f-7a86-4f76-ba37-467561cdd809 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:38.931020 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4847ac8-5d31-44c8-878f-6358ec271dc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:38.932468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4847ac8-5d31-44c8-878f-6358ec271dc9 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:38.949647 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1339/2656] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:39:38 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:39:38.950701 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cafd60f-7a86-4f76-ba37-467561cdd809 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:38.952004 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1318/2657] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:39:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:39:39.015829 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:39.032935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cafd60f-7a86-4f76-ba37-467561cdd809 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:39.039998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e18c1ef-c406-448e-a260-04a9b6584397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:39.041088 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18c1ef-c406-448e-a260-04a9b6584397 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:39.041678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18c1ef-c406-448e-a260-04a9b6584397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:39.042042 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18c1ef-c406-448e-a260-04a9b6584397 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:39.042739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e18c1ef-c406-448e-a260-04a9b6584397 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:39.044801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e18c1ef-c406-448e-a260-04a9b6584397 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:39.069822 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1340/2658] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:39:38 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:39:39.071503 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4847ac8-5d31-44c8-878f-6358ec271dc9 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:39.083030 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4847ac8-5d31-44c8-878f-6358ec271dc9 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:39.088440 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de50566-e890-4121-8a59-bd86ed3bf5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:39.089165 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de50566-e890-4121-8a59-bd86ed3bf5df admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:39.089512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de50566-e890-4121-8a59-bd86ed3bf5df admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:39.089881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de50566-e890-4121-8a59-bd86ed3bf5df admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:39.090681 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de50566-e890-4121-8a59-bd86ed3bf5df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:39.092341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de50566-e890-4121-8a59-bd86ed3bf5df admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:39.103562 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1319/2659] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:39:39.107650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e18c1ef-c406-448e-a260-04a9b6584397 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:39.117782 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e18c1ef-c406-448e-a260-04a9b6584397 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:39.122984 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc915b1-e560-4a5a-974e-6d53a849bf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:39.123783 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc915b1-e560-4a5a-974e-6d53a849bf9a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:39.124072 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc915b1-e560-4a5a-974e-6d53a849bf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:39.124451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc915b1-e560-4a5a-974e-6d53a849bf9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:39.125247 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc915b1-e560-4a5a-974e-6d53a849bf9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:39.127184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc915b1-e560-4a5a-974e-6d53a849bf9a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:39.144687 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1341/2660] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:39:39.150262 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de50566-e890-4121-8a59-bd86ed3bf5df admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:39.162624 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de50566-e890-4121-8a59-bd86ed3bf5df admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:39.167921 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff777f6-f8a6-4299-876a-c3c3673df6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:39.168830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff777f6-f8a6-4299-876a-c3c3673df6ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:39.169148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff777f6-f8a6-4299-876a-c3c3673df6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:39.169426 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff777f6-f8a6-4299-876a-c3c3673df6ee admin admin] PATH_INFO: `/v3/projects/0b4b6404624e413d94181a9be5f4b056/users/4c71647643aa4cd093bd3c57df50f099/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:39.186231 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff777f6-f8a6-4299-876a-c3c3673df6ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b4b6404624e413d94181a9be5f4b056, user_id=4c71647643aa4cd093bd3c57df50f099, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:39.188962 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff777f6-f8a6-4299-876a-c3c3673df6ee admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:39.203198 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1320/2661] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:39:39 2022] PUT /identity/v3/projects/0b4b6404624e413d94181a9be5f4b056/users/4c71647643aa4cd093bd3c57df50f099/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:39.214813 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6463f-acb4-469f-ab54-46a1deda69ce None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:39.215109 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6463f-acb4-469f-ab54-46a1deda69ce None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:39.215393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6463f-acb4-469f-ab54-46a1deda69ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:39.243838 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2ad6463f-acb4-469f-ab54-46a1deda69ce None None] MFA Rules not processed for user `4c71647643aa4cd093bd3c57df50f099`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:39.245081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2ad6463f-acb4-469f-ab54-46a1deda69ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:39.316220 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1342/2662] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:39.321611 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff777f6-f8a6-4299-876a-c3c3673df6ee admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:39.384501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff777f6-f8a6-4299-876a-c3c3673df6ee admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:39.389381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d4356-947f-4d9b-923c-239582beb170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:39.390355 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d4356-947f-4d9b-923c-239582beb170 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:39.390646 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d4356-947f-4d9b-923c-239582beb170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:39.390925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d4356-947f-4d9b-923c-239582beb170 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:39.391620 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535d4356-947f-4d9b-923c-239582beb170 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:39.393591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535d4356-947f-4d9b-923c-239582beb170 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:39.425906 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1321/2663] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:39:39 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:39:39.427226 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6463f-acb4-469f-ab54-46a1deda69ce None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:39.438803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6463f-acb4-469f-ab54-46a1deda69ce None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:39.504187 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b2a9f-23d9-406b-8686-8f39ffa49fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:39.504810 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b2a9f-23d9-406b-8686-8f39ffa49fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:39.505096 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b2a9f-23d9-406b-8686-8f39ffa49fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:39.505374 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b2a9f-23d9-406b-8686-8f39ffa49fc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:39.506094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b2a9f-23d9-406b-8686-8f39ffa49fc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:39.508698 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b2a9f-23d9-406b-8686-8f39ffa49fc5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:39.533455 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1343/2664] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:39:39 2022] POST /identity/v3/users => generated 298 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:39:39.537594 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d4356-947f-4d9b-923c-239582beb170 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:39.548189 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d4356-947f-4d9b-923c-239582beb170 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:39.552849 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824369f9-69b4-4c31-bd3a-83c06f2ec362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:39.553607 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824369f9-69b4-4c31-bd3a-83c06f2ec362 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:39.554032 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824369f9-69b4-4c31-bd3a-83c06f2ec362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:39.554341 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824369f9-69b4-4c31-bd3a-83c06f2ec362 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:39.555056 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824369f9-69b4-4c31-bd3a-83c06f2ec362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:39.556661 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824369f9-69b4-4c31-bd3a-83c06f2ec362 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:39.568401 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1322/2665] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:39:39.573144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b2a9f-23d9-406b-8686-8f39ffa49fc5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:39.583374 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b2a9f-23d9-406b-8686-8f39ffa49fc5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:39.588597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296bbe1-fbb2-42e1-ab7a-70f46bd2d451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:39.589507 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296bbe1-fbb2-42e1-ab7a-70f46bd2d451 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:39.589836 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296bbe1-fbb2-42e1-ab7a-70f46bd2d451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:39.590124 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296bbe1-fbb2-42e1-ab7a-70f46bd2d451 admin admin] PATH_INFO: `/v3/projects/3471603f308a42f8a75d238b598fabd5/users/a91115ac8f6d4eadaa3f415ffdf4800a/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:39.605284 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296bbe1-fbb2-42e1-ab7a-70f46bd2d451 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3471603f308a42f8a75d238b598fabd5, user_id=a91115ac8f6d4eadaa3f415ffdf4800a, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:39.607994 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296bbe1-fbb2-42e1-ab7a-70f46bd2d451 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:39.620449 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1344/2666] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:39:39 2022] PUT /identity/v3/projects/3471603f308a42f8a75d238b598fabd5/users/a91115ac8f6d4eadaa3f415ffdf4800a/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:39.624483 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824369f9-69b4-4c31-bd3a-83c06f2ec362 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:39.635058 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824369f9-69b4-4c31-bd3a-83c06f2ec362 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:39.638637 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:39.691656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83352cf-db2c-4b6b-afc4-bf900ca121de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:39.692314 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83352cf-db2c-4b6b-afc4-bf900ca121de admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:39.692602 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83352cf-db2c-4b6b-afc4-bf900ca121de admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:39.692876 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83352cf-db2c-4b6b-afc4-bf900ca121de admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:39.693606 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83352cf-db2c-4b6b-afc4-bf900ca121de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:39.695900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83352cf-db2c-4b6b-afc4-bf900ca121de admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:39.705639 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1323/2667] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:39:39.709753 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296bbe1-fbb2-42e1-ab7a-70f46bd2d451 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:39.720085 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296bbe1-fbb2-42e1-ab7a-70f46bd2d451 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:39.724732 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddead3-0d83-4ea8-b511-a7e7044b99ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:39.725662 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddead3-0d83-4ea8-b511-a7e7044b99ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:39.725926 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddead3-0d83-4ea8-b511-a7e7044b99ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:39.726209 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddead3-0d83-4ea8-b511-a7e7044b99ba admin admin] PATH_INFO: `/v3/projects/3471603f308a42f8a75d238b598fabd5/users/a91115ac8f6d4eadaa3f415ffdf4800a/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:39.728161 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddead3-0d83-4ea8-b511-a7e7044b99ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=3471603f308a42f8a75d238b598fabd5, user_id=a91115ac8f6d4eadaa3f415ffdf4800a, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:39.730835 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddead3-0d83-4ea8-b511-a7e7044b99ba admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:39.741668 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1345/2668] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:39:39 2022] PUT /identity/v3/projects/3471603f308a42f8a75d238b598fabd5/users/a91115ac8f6d4eadaa3f415ffdf4800a/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:39.751807 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e55322-cd53-456f-a533-10a9d786f8bc None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:39.752096 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e55322-cd53-456f-a533-10a9d786f8bc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:39.752377 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e55322-cd53-456f-a533-10a9d786f8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:39.778781 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-57e55322-cd53-456f-a533-10a9d786f8bc None None] MFA Rules not processed for user `a91115ac8f6d4eadaa3f415ffdf4800a`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:39.779924 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-57e55322-cd53-456f-a533-10a9d786f8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:39.848066 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1324/2669] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:39.859477 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddead3-0d83-4ea8-b511-a7e7044b99ba admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:39.861407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-88ddead3-0d83-4ea8-b511-a7e7044b99ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:39.875194 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:39.920247 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddead3-0d83-4ea8-b511-a7e7044b99ba admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:39.923931 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ff396-c2e1-4644-bfe4-f5215e9860ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:39.924569 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ff396-c2e1-4644-bfe4-f5215e9860ea service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:39.924859 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ff396-c2e1-4644-bfe4-f5215e9860ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:39.925141 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ff396-c2e1-4644-bfe4-f5215e9860ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:39.934594 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ff396-c2e1-4644-bfe4-f5215e9860ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:39.936034 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ff396-c2e1-4644-bfe4-f5215e9860ea service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:39.945968 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1346/2670] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:39:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:39:40.030826 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e55322-cd53-456f-a533-10a9d786f8bc None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:40.041016 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e55322-cd53-456f-a533-10a9d786f8bc None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:40.044653 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81bf0c-8745-4f70-8a84-cab934def7d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:40.045288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81bf0c-8745-4f70-8a84-cab934def7d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:40.045572 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81bf0c-8745-4f70-8a84-cab934def7d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:40.045870 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81bf0c-8745-4f70-8a84-cab934def7d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:40.103568 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d81bf0c-8745-4f70-8a84-cab934def7d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:40.105012 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d81bf0c-8745-4f70-8a84-cab934def7d7 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:40.113892 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1325/2671] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:39:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:39:40.373957 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ff396-c2e1-4644-bfe4-f5215e9860ea service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:40.435498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ff396-c2e1-4644-bfe4-f5215e9860ea service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:40.439990 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988fa3b-1f6c-4386-91c5-4862d5018d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:40.440943 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988fa3b-1f6c-4386-91c5-4862d5018d94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:40.441239 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988fa3b-1f6c-4386-91c5-4862d5018d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:40.441521 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988fa3b-1f6c-4386-91c5-4862d5018d94 admin admin] PATH_INFO: `/v3/users/4c71647643aa4cd093bd3c57df50f099` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:40.442763 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988fa3b-1f6c-4386-91c5-4862d5018d94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c71647643aa4cd093bd3c57df50f099)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:40.445280 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988fa3b-1f6c-4386-91c5-4862d5018d94 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:40.495916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7988fa3b-1f6c-4386-91c5-4862d5018d94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c71647643aa4cd093bd3c57df50f099'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:40.510977 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7988fa3b-1f6c-4386-91c5-4862d5018d94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c71647643aa4cd093bd3c57df50f099'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:40.511697 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7988fa3b-1f6c-4386-91c5-4862d5018d94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c71647643aa4cd093bd3c57df50f099'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:40.518624 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7988fa3b-1f6c-4386-91c5-4862d5018d94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c71647643aa4cd093bd3c57df50f099'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:40.519278 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-7988fa3b-1f6c-4386-91c5-4862d5018d94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c71647643aa4cd093bd3c57df50f099'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:40.533901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1347/2672] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:39:40 2022] DELETE /identity/v3/users/4c71647643aa4cd093bd3c57df50f099 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:40.541527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3c4ea-7576-4056-bd53-1e9c1068355f None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:40.541966 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3c4ea-7576-4056-bd53-1e9c1068355f None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:40.542424 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3c4ea-7576-4056-bd53-1e9c1068355f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:40.563313 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a7e3c4ea-7576-4056-bd53-1e9c1068355f None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:40.564525 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a7e3c4ea-7576-4056-bd53-1e9c1068355f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:40.586729 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:40.627466 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1326/2673] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:40.637543 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988fa3b-1f6c-4386-91c5-4862d5018d94 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:40.639112 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7988fa3b-1f6c-4386-91c5-4862d5018d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:40.673498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:40.698613 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988fa3b-1f6c-4386-91c5-4862d5018d94 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:40.701523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e63839-c07a-4145-a612-a6151f42eee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:40.702435 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e63839-c07a-4145-a612-a6151f42eee5 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:40.702714 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e63839-c07a-4145-a612-a6151f42eee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:40.702988 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e63839-c07a-4145-a612-a6151f42eee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:40.711950 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e63839-c07a-4145-a612-a6151f42eee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:40.713384 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e63839-c07a-4145-a612-a6151f42eee5 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:40.722960 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1348/2674] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:39:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:39:40.992033 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3c4ea-7576-4056-bd53-1e9c1068355f None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:40.993656 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a7e3c4ea-7576-4056-bd53-1e9c1068355f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:41.006475 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3c4ea-7576-4056-bd53-1e9c1068355f None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:41.011898 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b7afa-beb3-49ec-84d6-5a8880a644b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:41.012848 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b7afa-beb3-49ec-84d6-5a8880a644b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:41.013272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b7afa-beb3-49ec-84d6-5a8880a644b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:41.013554 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b7afa-beb3-49ec-84d6-5a8880a644b0 admin admin] PATH_INFO: `/v3/projects/0b4b6404624e413d94181a9be5f4b056` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:41.014802 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b7afa-beb3-49ec-84d6-5a8880a644b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b4b6404624e413d94181a9be5f4b056)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:41.017400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b7afa-beb3-49ec-84d6-5a8880a644b0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:41.069660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9f7b7afa-beb3-49ec-84d6-5a8880a644b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b4b6404624e413d94181a9be5f4b056', 'request_id': 'req-9f7b7afa-beb3-49ec-84d6-5a8880a644b0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:41.076060 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9f7b7afa-beb3-49ec-84d6-5a8880a644b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b4b6404624e413d94181a9be5f4b056', 'request_id': 'req-9f7b7afa-beb3-49ec-84d6-5a8880a644b0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:41.082089 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9f7b7afa-beb3-49ec-84d6-5a8880a644b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b4b6404624e413d94181a9be5f4b056', 'request_id': 'req-9f7b7afa-beb3-49ec-84d6-5a8880a644b0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:41.095119 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9f7b7afa-beb3-49ec-84d6-5a8880a644b0 admin admin] The token cache is being invalidate because project 0b4b6404624e413d94181a9be5f4b056 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:39:41.095416 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9f7b7afa-beb3-49ec-84d6-5a8880a644b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:41.096053 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-9f7b7afa-beb3-49ec-84d6-5a8880a644b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:41.097603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1327/2675] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:39:40 2022] DELETE /identity/v3/projects/0b4b6404624e413d94181a9be5f4b056 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:39:41.102883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e63839-c07a-4145-a612-a6151f42eee5 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:41.112624 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e63839-c07a-4145-a612-a6151f42eee5 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:41.170405 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d1fa0-8e0b-4c47-99bb-f64e892c2432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:41.171279 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d1fa0-8e0b-4c47-99bb-f64e892c2432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:41.171568 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d1fa0-8e0b-4c47-99bb-f64e892c2432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:41.171853 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d1fa0-8e0b-4c47-99bb-f64e892c2432 admin admin] PATH_INFO: `/v3/users/a91115ac8f6d4eadaa3f415ffdf4800a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:41.173144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d1fa0-8e0b-4c47-99bb-f64e892c2432 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a91115ac8f6d4eadaa3f415ffdf4800a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:41.175123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d1fa0-8e0b-4c47-99bb-f64e892c2432 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:41.222094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-958d1fa0-8e0b-4c47-99bb-f64e892c2432 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a91115ac8f6d4eadaa3f415ffdf4800a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:41.235593 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-958d1fa0-8e0b-4c47-99bb-f64e892c2432 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a91115ac8f6d4eadaa3f415ffdf4800a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:41.236298 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-958d1fa0-8e0b-4c47-99bb-f64e892c2432 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a91115ac8f6d4eadaa3f415ffdf4800a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:41.245375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-958d1fa0-8e0b-4c47-99bb-f64e892c2432 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a91115ac8f6d4eadaa3f415ffdf4800a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:41.246081 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-958d1fa0-8e0b-4c47-99bb-f64e892c2432 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a91115ac8f6d4eadaa3f415ffdf4800a'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:41.260320 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1349/2676] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:39:41 2022] DELETE /identity/v3/users/a91115ac8f6d4eadaa3f415ffdf4800a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:41.561028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b7afa-beb3-49ec-84d6-5a8880a644b0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:41.564536 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9f7b7afa-beb3-49ec-84d6-5a8880a644b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:41.586512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b7afa-beb3-49ec-84d6-5a8880a644b0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:41.652839 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e9f31-9040-4954-8342-f16a03d81fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:41.653868 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e9f31-9040-4954-8342-f16a03d81fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:41.654219 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e9f31-9040-4954-8342-f16a03d81fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:41.654673 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e9f31-9040-4954-8342-f16a03d81fb0 admin admin] PATH_INFO: `/v3/projects/3471603f308a42f8a75d238b598fabd5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:41.655922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16e9f31-9040-4954-8342-f16a03d81fb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3471603f308a42f8a75d238b598fabd5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:41.658019 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16e9f31-9040-4954-8342-f16a03d81fb0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:41.720426 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b16e9f31-9040-4954-8342-f16a03d81fb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3471603f308a42f8a75d238b598fabd5', 'request_id': 'req-b16e9f31-9040-4954-8342-f16a03d81fb0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:41.744722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:41.750958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b16e9f31-9040-4954-8342-f16a03d81fb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3471603f308a42f8a75d238b598fabd5', 'request_id': 'req-b16e9f31-9040-4954-8342-f16a03d81fb0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:41.759188 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b16e9f31-9040-4954-8342-f16a03d81fb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3471603f308a42f8a75d238b598fabd5', 'request_id': 'req-b16e9f31-9040-4954-8342-f16a03d81fb0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:41.774875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b16e9f31-9040-4954-8342-f16a03d81fb0 admin admin] The token cache is being invalidate because project 3471603f308a42f8a75d238b598fabd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:39:41.776204 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b16e9f31-9040-4954-8342-f16a03d81fb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:41.778020 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b16e9f31-9040-4954-8342-f16a03d81fb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:41.778785 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d1fa0-8e0b-4c47-99bb-f64e892c2432 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:41.780378 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-958d1fa0-8e0b-4c47-99bb-f64e892c2432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:41.781123 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1328/2677] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:39:41 2022] DELETE /identity/v3/projects/3471603f308a42f8a75d238b598fabd5 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:41.785569 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667d951-cea2-4c82-b766-266661446207 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:41.785958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667d951-cea2-4c82-b766-266661446207 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:41.786473 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667d951-cea2-4c82-b766-266661446207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:41.807682 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-1667d951-cea2-4c82-b766-266661446207 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:41.809274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1667d951-cea2-4c82-b766-266661446207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:41.888582 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:41.900801 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1329/2678] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:39:41.918381 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7bb9a-99c5-480f-8576-d03df4e41b9e None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:41.920759 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7bb9a-99c5-480f-8576-d03df4e41b9e None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:41.922500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7bb9a-99c5-480f-8576-d03df4e41b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:41.953266 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d1fa0-8e0b-4c47-99bb-f64e892c2432 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:41.966925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-86b7bb9a-99c5-480f-8576-d03df4e41b9e None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:41.969284 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-86b7bb9a-99c5-480f-8576-d03df4e41b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:41.982215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1330/2679] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:41.990190 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7bb9a-99c5-480f-8576-d03df4e41b9e None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:42.011050 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7bb9a-99c5-480f-8576-d03df4e41b9e None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:42.018357 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b6465-b193-4232-9cb0-55198793eeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:42.019398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b6465-b193-4232-9cb0-55198793eeb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:42.019694 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b6465-b193-4232-9cb0-55198793eeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:42.019976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b6465-b193-4232-9cb0-55198793eeb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:42.020711 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70b6465-b193-4232-9cb0-55198793eeb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:39:42.021055 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70b6465-b193-4232-9cb0-55198793eeb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:42.023658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70b6465-b193-4232-9cb0-55198793eeb1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:42.039842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1331/2680] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:39:42.041251 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b6465-b193-4232-9cb0-55198793eeb1 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:42.054764 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b6465-b193-4232-9cb0-55198793eeb1 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:42.055087 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540fc57c-ce6c-4e14-aebb-1ce88277b4dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:42.055757 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540fc57c-ce6c-4e14-aebb-1ce88277b4dc service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:42.056046 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540fc57c-ce6c-4e14-aebb-1ce88277b4dc service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:42.056470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540fc57c-ce6c-4e14-aebb-1ce88277b4dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:42.058352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-540fc57c-ce6c-4e14-aebb-1ce88277b4dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:42.059985 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166d37e-6498-44df-9c3a-daa3c37a3f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:42.060723 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166d37e-6498-44df-9c3a-daa3c37a3f3b admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:42.061007 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166d37e-6498-44df-9c3a-daa3c37a3f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:42.061292 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166d37e-6498-44df-9c3a-daa3c37a3f3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:42.062023 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0166d37e-6498-44df-9c3a-daa3c37a3f3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:42.063938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0166d37e-6498-44df-9c3a-daa3c37a3f3b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:42.074833 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540fc57c-ce6c-4e14-aebb-1ce88277b4dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:42.076383 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540fc57c-ce6c-4e14-aebb-1ce88277b4dc service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:42.088837 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1350/2681] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:39:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:39:42.101444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1332/2682] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:39:42 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:39:42.102946 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540fc57c-ce6c-4e14-aebb-1ce88277b4dc service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:42.115335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540fc57c-ce6c-4e14-aebb-1ce88277b4dc service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:42.181345 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22669040-3d0f-4e41-a144-5005945aafa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:42.181965 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22669040-3d0f-4e41-a144-5005945aafa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:42.182254 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22669040-3d0f-4e41-a144-5005945aafa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:42.182536 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22669040-3d0f-4e41-a144-5005945aafa3 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:42.183259 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22669040-3d0f-4e41-a144-5005945aafa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:42.185196 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22669040-3d0f-4e41-a144-5005945aafa3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:42.210644 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1351/2683] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:39:42 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:39:42.212546 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166d37e-6498-44df-9c3a-daa3c37a3f3b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:42.224396 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166d37e-6498-44df-9c3a-daa3c37a3f3b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:42.230522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eafa9b-8db5-493d-9a51-23e49ba430f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:42.231480 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eafa9b-8db5-493d-9a51-23e49ba430f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:42.231650 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eafa9b-8db5-493d-9a51-23e49ba430f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:42.231963 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eafa9b-8db5-493d-9a51-23e49ba430f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:42.232768 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eafa9b-8db5-493d-9a51-23e49ba430f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:42.234426 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eafa9b-8db5-493d-9a51-23e49ba430f9 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:42.248219 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1333/2684] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:39:42.250658 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22669040-3d0f-4e41-a144-5005945aafa3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:42.261671 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22669040-3d0f-4e41-a144-5005945aafa3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:42.266494 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e84bbe-feba-44d3-9a57-47f91d88cd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:42.267174 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e84bbe-feba-44d3-9a57-47f91d88cd01 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:42.267519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e84bbe-feba-44d3-9a57-47f91d88cd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:42.267808 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e84bbe-feba-44d3-9a57-47f91d88cd01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:42.268654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e84bbe-feba-44d3-9a57-47f91d88cd01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:42.270308 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e84bbe-feba-44d3-9a57-47f91d88cd01 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:42.284523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1352/2685] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:39:42.285756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eafa9b-8db5-493d-9a51-23e49ba430f9 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:42.297236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eafa9b-8db5-493d-9a51-23e49ba430f9 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:42.302191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae299c-3879-4700-a28e-87e2dfa7e388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:42.303101 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae299c-3879-4700-a28e-87e2dfa7e388 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:42.303621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae299c-3879-4700-a28e-87e2dfa7e388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:42.304107 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae299c-3879-4700-a28e-87e2dfa7e388 admin admin] PATH_INFO: `/v3/projects/d22c05ce9b7b471386403892b8bb4b06/users/da4b24981e4048f5bf296aad4c0faa66/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:42.320665 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae299c-3879-4700-a28e-87e2dfa7e388 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d22c05ce9b7b471386403892b8bb4b06, user_id=da4b24981e4048f5bf296aad4c0faa66, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:42.323584 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae299c-3879-4700-a28e-87e2dfa7e388 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:42.340286 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1334/2686] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:39:42 2022] PUT /identity/v3/projects/d22c05ce9b7b471386403892b8bb4b06/users/da4b24981e4048f5bf296aad4c0faa66/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:42.406996 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66addd45-7352-48da-9783-6b51ffae8295 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:42.407406 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66addd45-7352-48da-9783-6b51ffae8295 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:42.407688 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66addd45-7352-48da-9783-6b51ffae8295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:42.435019 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-66addd45-7352-48da-9783-6b51ffae8295 None None] MFA Rules not processed for user `da4b24981e4048f5bf296aad4c0faa66`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:42.436378 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-66addd45-7352-48da-9783-6b51ffae8295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:42.482938 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:42.505464 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1353/2687] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:42.517373 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae299c-3879-4700-a28e-87e2dfa7e388 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:42.519619 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a0ae299c-3879-4700-a28e-87e2dfa7e388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:42.579090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae299c-3879-4700-a28e-87e2dfa7e388 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:42.582885 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef875a3-674c-4a49-bf20-dc47c4faf8b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:42.583592 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef875a3-674c-4a49-bf20-dc47c4faf8b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:42.583880 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef875a3-674c-4a49-bf20-dc47c4faf8b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:42.584155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef875a3-674c-4a49-bf20-dc47c4faf8b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:42.594314 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef875a3-674c-4a49-bf20-dc47c4faf8b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:42.595765 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef875a3-674c-4a49-bf20-dc47c4faf8b4 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:42.606888 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1335/2688] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:39:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:39:42.626836 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66addd45-7352-48da-9783-6b51ffae8295 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:42.690386 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66addd45-7352-48da-9783-6b51ffae8295 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:42.695266 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53cdf3d-68c5-498b-8850-c8c5345bd27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:42.696142 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53cdf3d-68c5-498b-8850-c8c5345bd27f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:42.696425 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53cdf3d-68c5-498b-8850-c8c5345bd27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:42.696699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53cdf3d-68c5-498b-8850-c8c5345bd27f admin admin] PATH_INFO: `/v3/users/da4b24981e4048f5bf296aad4c0faa66` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:42.697983 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53cdf3d-68c5-498b-8850-c8c5345bd27f admin admin] RBAC: Authorizing `identity:delete_user(user_id=da4b24981e4048f5bf296aad4c0faa66)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:42.699928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53cdf3d-68c5-498b-8850-c8c5345bd27f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:42.756901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e53cdf3d-68c5-498b-8850-c8c5345bd27f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da4b24981e4048f5bf296aad4c0faa66'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:42.769314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e53cdf3d-68c5-498b-8850-c8c5345bd27f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da4b24981e4048f5bf296aad4c0faa66'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:42.770678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e53cdf3d-68c5-498b-8850-c8c5345bd27f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da4b24981e4048f5bf296aad4c0faa66'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:42.780993 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e53cdf3d-68c5-498b-8850-c8c5345bd27f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da4b24981e4048f5bf296aad4c0faa66'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:42.783122 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-e53cdf3d-68c5-498b-8850-c8c5345bd27f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da4b24981e4048f5bf296aad4c0faa66'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:42.799159 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1354/2689] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:39:42 2022] DELETE /identity/v3/users/da4b24981e4048f5bf296aad4c0faa66 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:42.804040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b939c1-961e-4fdf-ae4f-fb82020779fc None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:42.804482 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b939c1-961e-4fdf-ae4f-fb82020779fc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:42.804857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b939c1-961e-4fdf-ae4f-fb82020779fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:42.817904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:42.825158 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-f2b939c1-961e-4fdf-ae4f-fb82020779fc None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:42.826821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f2b939c1-961e-4fdf-ae4f-fb82020779fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:42.900419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1336/2690] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:42.912765 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53cdf3d-68c5-498b-8850-c8c5345bd27f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:42.914477 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e53cdf3d-68c5-498b-8850-c8c5345bd27f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:42.995421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53cdf3d-68c5-498b-8850-c8c5345bd27f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:42.999509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378c8a4-581f-43df-b0ac-6b3036efa995 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:43.000143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378c8a4-581f-43df-b0ac-6b3036efa995 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:43.000432 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378c8a4-581f-43df-b0ac-6b3036efa995 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:43.000712 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378c8a4-581f-43df-b0ac-6b3036efa995 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:43.011743 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378c8a4-581f-43df-b0ac-6b3036efa995 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:43.013609 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378c8a4-581f-43df-b0ac-6b3036efa995 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:43.025851 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1355/2691] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:39:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:39:43.310485 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b939c1-961e-4fdf-ae4f-fb82020779fc None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:43.312781 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-f2b939c1-961e-4fdf-ae4f-fb82020779fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:43.342187 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b939c1-961e-4fdf-ae4f-fb82020779fc None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:43.348380 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b49ec-7ae5-4d9e-8f67-8e927912a2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:43.349321 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b49ec-7ae5-4d9e-8f67-8e927912a2dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:43.349638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b49ec-7ae5-4d9e-8f67-8e927912a2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:43.350453 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b49ec-7ae5-4d9e-8f67-8e927912a2dd admin admin] PATH_INFO: `/v3/projects/d22c05ce9b7b471386403892b8bb4b06` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:43.352290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530b49ec-7ae5-4d9e-8f67-8e927912a2dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d22c05ce9b7b471386403892b8bb4b06)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:43.354258 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530b49ec-7ae5-4d9e-8f67-8e927912a2dd admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:43.404700 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-530b49ec-7ae5-4d9e-8f67-8e927912a2dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd22c05ce9b7b471386403892b8bb4b06', 'request_id': 'req-530b49ec-7ae5-4d9e-8f67-8e927912a2dd'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:43.410941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-530b49ec-7ae5-4d9e-8f67-8e927912a2dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd22c05ce9b7b471386403892b8bb4b06', 'request_id': 'req-530b49ec-7ae5-4d9e-8f67-8e927912a2dd'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:43.417564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-530b49ec-7ae5-4d9e-8f67-8e927912a2dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd22c05ce9b7b471386403892b8bb4b06', 'request_id': 'req-530b49ec-7ae5-4d9e-8f67-8e927912a2dd'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:43.434868 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-530b49ec-7ae5-4d9e-8f67-8e927912a2dd admin admin] The token cache is being invalidate because project d22c05ce9b7b471386403892b8bb4b06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:39:43.435308 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-530b49ec-7ae5-4d9e-8f67-8e927912a2dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:43.436032 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-530b49ec-7ae5-4d9e-8f67-8e927912a2dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:43.441039 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1337/2692] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:39:43 2022] DELETE /identity/v3/projects/d22c05ce9b7b471386403892b8bb4b06 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:43.442998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70478477-4023-410f-8beb-4f22dfa18fe4 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:43.443421 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70478477-4023-410f-8beb-4f22dfa18fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:43.444239 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70478477-4023-410f-8beb-4f22dfa18fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:43.462857 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-70478477-4023-410f-8beb-4f22dfa18fe4 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:43.464172 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-70478477-4023-410f-8beb-4f22dfa18fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:43.468597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:43.530657 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1356/2693] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:43.538979 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a29126-6783-47f7-9a59-41ef55f31edc None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:43.539264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a29126-6783-47f7-9a59-41ef55f31edc None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:43.539542 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a29126-6783-47f7-9a59-41ef55f31edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:43.558274 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-16a29126-6783-47f7-9a59-41ef55f31edc None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:43.559598 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-16a29126-6783-47f7-9a59-41ef55f31edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:43.568593 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1338/2694] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:39:43.572439 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70478477-4023-410f-8beb-4f22dfa18fe4 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:43.582897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70478477-4023-410f-8beb-4f22dfa18fe4 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:43.596570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b7c22-b2e0-4357-8853-e564ad872cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:43.597351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b7c22-b2e0-4357-8853-e564ad872cdc admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:43.597689 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b7c22-b2e0-4357-8853-e564ad872cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:43.598121 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b7c22-b2e0-4357-8853-e564ad872cdc admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:43.598609 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b7c22-b2e0-4357-8853-e564ad872cdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:39:43.599008 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b7c22-b2e0-4357-8853-e564ad872cdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:43.601479 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b7c22-b2e0-4357-8853-e564ad872cdc admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:43.615265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1357/2695] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:39:43.620810 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a29126-6783-47f7-9a59-41ef55f31edc None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:43.636277 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a29126-6783-47f7-9a59-41ef55f31edc None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:43.641022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3a6cb-b8bd-4377-b240-239d6b07f3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:43.641715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3a6cb-b8bd-4377-b240-239d6b07f3d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:43.642004 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3a6cb-b8bd-4377-b240-239d6b07f3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:43.642286 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3a6cb-b8bd-4377-b240-239d6b07f3d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:43.642981 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3a6cb-b8bd-4377-b240-239d6b07f3d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:43.645272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3a6cb-b8bd-4377-b240-239d6b07f3d4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:43.673445 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1339/2696] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:39:43 2022] POST /identity/v3/projects => generated 397 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:39:43.678225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b7c22-b2e0-4357-8853-e564ad872cdc admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:43.688279 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b7c22-b2e0-4357-8853-e564ad872cdc admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:43.744060 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e856f-ee73-4a62-8057-8d8d596dcc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:43.744648 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e856f-ee73-4a62-8057-8d8d596dcc86 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:43.744934 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e856f-ee73-4a62-8057-8d8d596dcc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:43.745228 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e856f-ee73-4a62-8057-8d8d596dcc86 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:43.745946 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e856f-ee73-4a62-8057-8d8d596dcc86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:43.747865 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e856f-ee73-4a62-8057-8d8d596dcc86 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:43.768237 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1358/2697] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:39:43 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:39:43.772490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3a6cb-b8bd-4377-b240-239d6b07f3d4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:43.782863 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3a6cb-b8bd-4377-b240-239d6b07f3d4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:43.788740 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc5655-0be8-43e4-80eb-6b0ebca605e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:43.789648 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc5655-0be8-43e4-80eb-6b0ebca605e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:43.789992 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc5655-0be8-43e4-80eb-6b0ebca605e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:43.790362 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc5655-0be8-43e4-80eb-6b0ebca605e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:43.791236 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc5655-0be8-43e4-80eb-6b0ebca605e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:43.793004 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc5655-0be8-43e4-80eb-6b0ebca605e5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:43.804273 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1340/2698] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:39:43.808219 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e856f-ee73-4a62-8057-8d8d596dcc86 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:43.818511 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e856f-ee73-4a62-8057-8d8d596dcc86 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:43.823637 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca69e53-bc2e-4d5d-8e7e-45c15f31ccd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:43.824292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca69e53-bc2e-4d5d-8e7e-45c15f31ccd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:43.824582 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca69e53-bc2e-4d5d-8e7e-45c15f31ccd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:43.824914 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca69e53-bc2e-4d5d-8e7e-45c15f31ccd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:43.825675 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca69e53-bc2e-4d5d-8e7e-45c15f31ccd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:43.827531 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca69e53-bc2e-4d5d-8e7e-45c15f31ccd0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:43.838199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1359/2699] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:39:43.842372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc5655-0be8-43e4-80eb-6b0ebca605e5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:43.853934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc5655-0be8-43e4-80eb-6b0ebca605e5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:43.858614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d68f6-5fbd-4415-acfe-7829667cf80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:43.859611 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d68f6-5fbd-4415-acfe-7829667cf80d admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:43.859895 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d68f6-5fbd-4415-acfe-7829667cf80d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:43.860178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d68f6-5fbd-4415-acfe-7829667cf80d admin admin] PATH_INFO: `/v3/projects/ccda19fcc3404d1a801c2293193e7bb5/users/c26949cda70d4d33b96f0e570b7a898d/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:43.875899 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d68f6-5fbd-4415-acfe-7829667cf80d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccda19fcc3404d1a801c2293193e7bb5, user_id=c26949cda70d4d33b96f0e570b7a898d, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:43.878638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d68f6-5fbd-4415-acfe-7829667cf80d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:43.891893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1341/2700] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:39:43 2022] PUT /identity/v3/projects/ccda19fcc3404d1a801c2293193e7bb5/users/c26949cda70d4d33b96f0e570b7a898d/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:39:43.903261 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2963a-4f1a-494b-9778-b4826cd5c985 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:43.903563 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2963a-4f1a-494b-9778-b4826cd5c985 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:43.903836 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2963a-4f1a-494b-9778-b4826cd5c985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:43.937868 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-35a2963a-4f1a-494b-9778-b4826cd5c985 None None] MFA Rules not processed for user `c26949cda70d4d33b96f0e570b7a898d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:43.939109 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-35a2963a-4f1a-494b-9778-b4826cd5c985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:43.955932 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d68f6-5fbd-4415-acfe-7829667cf80d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:43.957783 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4c7d68f6-5fbd-4415-acfe-7829667cf80d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:43.988551 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:44.011794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1360/2701] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:44.017645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d68f6-5fbd-4415-acfe-7829667cf80d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:44.021287 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83e52a-1aa7-45e4-8e0e-b66c572985ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:44.022164 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83e52a-1aa7-45e4-8e0e-b66c572985ec service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:44.022591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83e52a-1aa7-45e4-8e0e-b66c572985ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:44.022893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83e52a-1aa7-45e4-8e0e-b66c572985ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:44.024791 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7a83e52a-1aa7-45e4-8e0e-b66c572985ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:44.026629 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2963a-4f1a-494b-9778-b4826cd5c985 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:44.028104 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-35a2963a-4f1a-494b-9778-b4826cd5c985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:44.036215 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:44.041755 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2963a-4f1a-494b-9778-b4826cd5c985 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:44.046485 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad881fc-0954-40f7-8021-1e6843156331 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:44.047002 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad881fc-0954-40f7-8021-1e6843156331 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:44.047309 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad881fc-0954-40f7-8021-1e6843156331 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:44.047788 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad881fc-0954-40f7-8021-1e6843156331 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:44.057606 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad881fc-0954-40f7-8021-1e6843156331 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:44.059094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad881fc-0954-40f7-8021-1e6843156331 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:44.073140 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1361/2702] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:39:44 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:39:44.087974 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a83e52a-1aa7-45e4-8e0e-b66c572985ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:44.089424 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a83e52a-1aa7-45e4-8e0e-b66c572985ec service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:44.099419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1342/2703] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:39:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:39:51.156125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad881fc-0954-40f7-8021-1e6843156331 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:51.157528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-cad881fc-0954-40f7-8021-1e6843156331 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:51.223616 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad881fc-0954-40f7-8021-1e6843156331 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:51.228004 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebfd5d-cbed-47c2-b60b-fcda7612fc55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:51.228675 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebfd5d-cbed-47c2-b60b-fcda7612fc55 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:51.228964 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebfd5d-cbed-47c2-b60b-fcda7612fc55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:51.229245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebfd5d-cbed-47c2-b60b-fcda7612fc55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:51.231049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b6ebfd5d-cbed-47c2-b60b-fcda7612fc55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:51.293375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebfd5d-cbed-47c2-b60b-fcda7612fc55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:51.294848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebfd5d-cbed-47c2-b60b-fcda7612fc55 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:51.305355 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1362/2704] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:39:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:39:54.094475 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=36 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:54.494160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83e52a-1aa7-45e4-8e0e-b66c572985ec service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:54.495616 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7a83e52a-1aa7-45e4-8e0e-b66c572985ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:54.509473 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83e52a-1aa7-45e4-8e0e-b66c572985ec service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:54.514317 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03482609-ad10-4a79-8846-791b8bbc2135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:54.515187 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03482609-ad10-4a79-8846-791b8bbc2135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:54.515472 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03482609-ad10-4a79-8846-791b8bbc2135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:54.515751 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03482609-ad10-4a79-8846-791b8bbc2135 admin admin] PATH_INFO: `/v3/users/f13e9e1bf5084d08919845767c70c3f7` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:54.516954 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03482609-ad10-4a79-8846-791b8bbc2135 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f13e9e1bf5084d08919845767c70c3f7)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:54.519068 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03482609-ad10-4a79-8846-791b8bbc2135 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:54.566327 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-03482609-ad10-4a79-8846-791b8bbc2135 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f13e9e1bf5084d08919845767c70c3f7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:54.567014 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-03482609-ad10-4a79-8846-791b8bbc2135 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f13e9e1bf5084d08919845767c70c3f7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:54.578967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-03482609-ad10-4a79-8846-791b8bbc2135 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f13e9e1bf5084d08919845767c70c3f7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:54.586775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-03482609-ad10-4a79-8846-791b8bbc2135 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f13e9e1bf5084d08919845767c70c3f7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:54.587421 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-03482609-ad10-4a79-8846-791b8bbc2135 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f13e9e1bf5084d08919845767c70c3f7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:54.600076 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1343/2705] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:39:54 2022] DELETE /identity/v3/users/f13e9e1bf5084d08919845767c70c3f7 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:54.778444 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebfd5d-cbed-47c2-b60b-fcda7612fc55 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:54.789603 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebfd5d-cbed-47c2-b60b-fcda7612fc55 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:54.847086 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738366db-26ea-4574-839d-e97dd82b8978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:54.847938 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738366db-26ea-4574-839d-e97dd82b8978 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:54.848250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738366db-26ea-4574-839d-e97dd82b8978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:54.849053 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738366db-26ea-4574-839d-e97dd82b8978 admin admin] PATH_INFO: `/v3/projects/f643222ab6194191979d2e884f5ed75a` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:54.850254 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738366db-26ea-4574-839d-e97dd82b8978 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f643222ab6194191979d2e884f5ed75a)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:54.852175 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738366db-26ea-4574-839d-e97dd82b8978 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:54.899051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:54.903129 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-738366db-26ea-4574-839d-e97dd82b8978 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f643222ab6194191979d2e884f5ed75a', 'request_id': 'req-738366db-26ea-4574-839d-e97dd82b8978'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:54.909552 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-738366db-26ea-4574-839d-e97dd82b8978 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f643222ab6194191979d2e884f5ed75a', 'request_id': 'req-738366db-26ea-4574-839d-e97dd82b8978'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:54.915603 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-738366db-26ea-4574-839d-e97dd82b8978 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f643222ab6194191979d2e884f5ed75a', 'request_id': 'req-738366db-26ea-4574-839d-e97dd82b8978'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:54.928405 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-738366db-26ea-4574-839d-e97dd82b8978 admin admin] The token cache is being invalidate because project f643222ab6194191979d2e884f5ed75a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:39:54.928720 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-738366db-26ea-4574-839d-e97dd82b8978 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:54.929333 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-738366db-26ea-4574-839d-e97dd82b8978 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:39:54.930867 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1363/2706] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:39:54 2022] DELETE /identity/v3/projects/f643222ab6194191979d2e884f5ed75a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:39:54.935842 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc133277-7f65-4340-bb00-c1c6fea9e4f7 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:54.936135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc133277-7f65-4340-bb00-c1c6fea9e4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:54.936418 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc133277-7f65-4340-bb00-c1c6fea9e4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:54.955022 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fc133277-7f65-4340-bb00-c1c6fea9e4f7 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:54.956239 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fc133277-7f65-4340-bb00-c1c6fea9e4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:55.018482 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1344/2707] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:55.026370 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ff363-329e-4154-b6b0-413860823d4d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:55.026751 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ff363-329e-4154-b6b0-413860823d4d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:55.027129 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ff363-329e-4154-b6b0-413860823d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:55.048155 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-f53ff363-329e-4154-b6b0-413860823d4d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:55.049561 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f53ff363-329e-4154-b6b0-413860823d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:55.066548 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1364/2708] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:39:55.070741 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc133277-7f65-4340-bb00-c1c6fea9e4f7 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:55.080651 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc133277-7f65-4340-bb00-c1c6fea9e4f7 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:55.085451 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8b8ad-e288-4abe-b09f-d970158a4cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:55.086159 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8b8ad-e288-4abe-b09f-d970158a4cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:55.086446 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8b8ad-e288-4abe-b09f-d970158a4cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:55.086738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8b8ad-e288-4abe-b09f-d970158a4cd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:55.087548 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8b8ad-e288-4abe-b09f-d970158a4cd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:39:55.087747 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8b8ad-e288-4abe-b09f-d970158a4cd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:55.089364 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8b8ad-e288-4abe-b09f-d970158a4cd0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:55.108397 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1345/2709] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:39:55.110118 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ff363-329e-4154-b6b0-413860823d4d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:55.119951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ff363-329e-4154-b6b0-413860823d4d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:55.130622 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84100c-9f86-4fe9-bd95-3c5eb31144c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:55.131299 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84100c-9f86-4fe9-bd95-3c5eb31144c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:55.131588 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84100c-9f86-4fe9-bd95-3c5eb31144c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:55.131921 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84100c-9f86-4fe9-bd95-3c5eb31144c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:55.132658 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84100c-9f86-4fe9-bd95-3c5eb31144c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:55.134569 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84100c-9f86-4fe9-bd95-3c5eb31144c8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:55.161091 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1365/2710] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:39:55 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:39:55.165508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8b8ad-e288-4abe-b09f-d970158a4cd0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:55.175411 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8b8ad-e288-4abe-b09f-d970158a4cd0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:55.231715 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a00af2-0fe5-45c6-b560-0b6f9a69b315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:55.232379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a00af2-0fe5-45c6-b560-0b6f9a69b315 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:55.232664 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a00af2-0fe5-45c6-b560-0b6f9a69b315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:55.232944 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a00af2-0fe5-45c6-b560-0b6f9a69b315 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:55.233664 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a00af2-0fe5-45c6-b560-0b6f9a69b315 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:55.235593 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a00af2-0fe5-45c6-b560-0b6f9a69b315 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:55.242179 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:55.256888 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1346/2711] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:39:55 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:39:55.260975 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84100c-9f86-4fe9-bd95-3c5eb31144c8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:55.272059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84100c-9f86-4fe9-bd95-3c5eb31144c8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:55.276678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245d63b-437b-4190-8246-01188e70ecdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:55.277332 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245d63b-437b-4190-8246-01188e70ecdd admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:55.277648 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245d63b-437b-4190-8246-01188e70ecdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:55.277934 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245d63b-437b-4190-8246-01188e70ecdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:55.278658 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4245d63b-437b-4190-8246-01188e70ecdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:55.280250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4245d63b-437b-4190-8246-01188e70ecdd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:55.290570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1366/2712] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:39:55.294568 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a00af2-0fe5-45c6-b560-0b6f9a69b315 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:55.304378 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a00af2-0fe5-45c6-b560-0b6f9a69b315 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:55.308971 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81329a3f-5056-438c-bdeb-785aed1523d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:55.309644 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81329a3f-5056-438c-bdeb-785aed1523d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:55.309928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81329a3f-5056-438c-bdeb-785aed1523d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:55.310215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81329a3f-5056-438c-bdeb-785aed1523d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:55.310938 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81329a3f-5056-438c-bdeb-785aed1523d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:55.312525 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81329a3f-5056-438c-bdeb-785aed1523d4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:55.323035 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1347/2713] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:39:55.327287 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245d63b-437b-4190-8246-01188e70ecdd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:55.337788 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245d63b-437b-4190-8246-01188e70ecdd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:55.342589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98f748-8b92-4de2-9dc4-8d9648fe3e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:55.343487 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98f748-8b92-4de2-9dc4-8d9648fe3e76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:55.343769 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98f748-8b92-4de2-9dc4-8d9648fe3e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:55.344046 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98f748-8b92-4de2-9dc4-8d9648fe3e76 admin admin] PATH_INFO: `/v3/projects/b8227be3d9a044ed888b2f54abd9463d/users/9531d18446eb4488bbd1511cb3412925/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:55.358181 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e98f748-8b92-4de2-9dc4-8d9648fe3e76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8227be3d9a044ed888b2f54abd9463d, user_id=9531d18446eb4488bbd1511cb3412925, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:55.360828 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e98f748-8b92-4de2-9dc4-8d9648fe3e76 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:55.374286 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1367/2714] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:39:55 2022] PUT /identity/v3/projects/b8227be3d9a044ed888b2f54abd9463d/users/9531d18446eb4488bbd1511cb3412925/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:39:55.379357 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2470bd-5a73-4be5-a2c0-1036d9b20616 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:55.380270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2470bd-5a73-4be5-a2c0-1036d9b20616 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:55.380657 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2470bd-5a73-4be5-a2c0-1036d9b20616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:55.398091 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-6f2470bd-5a73-4be5-a2c0-1036d9b20616 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:55.399299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-6f2470bd-5a73-4be5-a2c0-1036d9b20616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:55.457460 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1348/2715] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:55.466931 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98f748-8b92-4de2-9dc4-8d9648fe3e76 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:55.468659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3e98f748-8b92-4de2-9dc4-8d9648fe3e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:55.525381 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98f748-8b92-4de2-9dc4-8d9648fe3e76 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:55.529085 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd60c951-9d3c-437a-8a76-460a63750a50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:55.529765 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60c951-9d3c-437a-8a76-460a63750a50 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:55.530052 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60c951-9d3c-437a-8a76-460a63750a50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:55.530330 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60c951-9d3c-437a-8a76-460a63750a50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:55.539377 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd60c951-9d3c-437a-8a76-460a63750a50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:55.540803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd60c951-9d3c-437a-8a76-460a63750a50 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:55.550409 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1368/2716] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:39:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:39:59.417468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66166af-1549-4351-b9d4-83983eb38032 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:59.418112 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66166af-1549-4351-b9d4-83983eb38032 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:59.418112 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66166af-1549-4351-b9d4-83983eb38032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:59.466809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-b66166af-1549-4351-b9d4-83983eb38032 None None] MFA Rules not processed for user `31c0594d0be84154a601d625bca699cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:59.468708 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b66166af-1549-4351-b9d4-83983eb38032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:59.540918 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1349/2717] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:59.668807 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c7a2e-3061-446e-b850-9e7e260ff015 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:59.669932 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c7a2e-3061-446e-b850-9e7e260ff015 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:59.669932 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c7a2e-3061-446e-b850-9e7e260ff015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:59.702057 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6a2c7a2e-3061-446e-b850-9e7e260ff015 None None] MFA Rules not processed for user `9531d18446eb4488bbd1511cb3412925`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:39:59.703291 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6a2c7a2e-3061-446e-b850-9e7e260ff015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:59.776810 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:39:59.782595 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1369/2718] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:39:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:39:59.796418 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66166af-1549-4351-b9d4-83983eb38032 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:59.797843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b66166af-1549-4351-b9d4-83983eb38032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:59.895123 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66166af-1549-4351-b9d4-83983eb38032 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:59.901816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02227dad-7f58-450b-95e9-0b755be0bcfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:59.902612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02227dad-7f58-450b-95e9-0b755be0bcfa service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:59.902998 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02227dad-7f58-450b-95e9-0b755be0bcfa service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:59.903400 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02227dad-7f58-450b-95e9-0b755be0bcfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:59.913372 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02227dad-7f58-450b-95e9-0b755be0bcfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:59.915105 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02227dad-7f58-450b-95e9-0b755be0bcfa service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:39:59.926086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1350/2719] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:39:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:39:59.949618 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c7a2e-3061-446e-b850-9e7e260ff015 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:39:59.951059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6a2c7a2e-3061-446e-b850-9e7e260ff015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:59.965665 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c7a2e-3061-446e-b850-9e7e260ff015 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:39:59.970123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916b826-b3f7-400c-aaba-fd29da60be5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:39:59.971163 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916b826-b3f7-400c-aaba-fd29da60be5d service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:39:59.971986 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916b826-b3f7-400c-aaba-fd29da60be5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:39:59.972873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916b826-b3f7-400c-aaba-fd29da60be5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:39:59.975702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4916b826-b3f7-400c-aaba-fd29da60be5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:39:59.995150 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4916b826-b3f7-400c-aaba-fd29da60be5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:39:59.995150 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4916b826-b3f7-400c-aaba-fd29da60be5d service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:40:00.002062 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1370/2720] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:39:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:40:00.674821 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02227dad-7f58-450b-95e9-0b755be0bcfa service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:40:00.677218 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-02227dad-7f58-450b-95e9-0b755be0bcfa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:40:00.696724 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:40:00.774011 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02227dad-7f58-450b-95e9-0b755be0bcfa service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:40:00.779257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d833f229-6da5-4175-a2bd-844e3053ac86 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49325c1733174e0c98b3475a10c9b5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:40:00.779923 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833f229-6da5-4175-a2bd-844e3053ac86 service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:40:00.780199 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833f229-6da5-4175-a2bd-844e3053ac86 service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:40:00.780480 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833f229-6da5-4175-a2bd-844e3053ac86 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:40:00.782336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-d833f229-6da5-4175-a2bd-844e3053ac86 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:40:00.795571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d833f229-6da5-4175-a2bd-844e3053ac86 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:40:00.797408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d833f229-6da5-4175-a2bd-844e3053ac86 service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:40:00.809180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1351/2721] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:40:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:40:01.728043 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916b826-b3f7-400c-aaba-fd29da60be5d service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:40:01.730270 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4916b826-b3f7-400c-aaba-fd29da60be5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:40:01.769407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916b826-b3f7-400c-aaba-fd29da60be5d service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:40:01.786712 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3ce19-64bd-4ca6-b861-2aac291ee2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:40:01.788370 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3ce19-64bd-4ca6-b861-2aac291ee2d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:40:01.788870 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3ce19-64bd-4ca6-b861-2aac291ee2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:40:01.789367 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3ce19-64bd-4ca6-b861-2aac291ee2d8 admin admin] PATH_INFO: `/v3/users/c26949cda70d4d33b96f0e570b7a898d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:40:01.796500 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c3ce19-64bd-4ca6-b861-2aac291ee2d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c26949cda70d4d33b96f0e570b7a898d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:40:01.798851 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c3ce19-64bd-4ca6-b861-2aac291ee2d8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:40:01.836538 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66973f-43d8-4bb0-98da-61f037673d40 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:40:01.837544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66973f-43d8-4bb0-98da-61f037673d40 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:40:01.838015 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66973f-43d8-4bb0-98da-61f037673d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:40:01.859578 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-ce66973f-43d8-4bb0-98da-61f037673d40 None None] MFA Rules not processed for user `9531d18446eb4488bbd1511cb3412925`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:40:01.861037 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ce66973f-43d8-4bb0-98da-61f037673d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:40:01.870200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1352/2722] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:40:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:40:01.895645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-69c3ce19-64bd-4ca6-b861-2aac291ee2d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c26949cda70d4d33b96f0e570b7a898d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:01.913154 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-69c3ce19-64bd-4ca6-b861-2aac291ee2d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c26949cda70d4d33b96f0e570b7a898d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:01.917454 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-69c3ce19-64bd-4ca6-b861-2aac291ee2d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c26949cda70d4d33b96f0e570b7a898d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:01.927856 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-69c3ce19-64bd-4ca6-b861-2aac291ee2d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c26949cda70d4d33b96f0e570b7a898d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:01.928666 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-69c3ce19-64bd-4ca6-b861-2aac291ee2d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c26949cda70d4d33b96f0e570b7a898d'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:01.947356 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1371/2723] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:40:01 2022] DELETE /identity/v3/users/c26949cda70d4d33b96f0e570b7a898d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:40:01.952591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1edae2-103c-4d2c-8e79-c29524a82574 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:40:01.952716 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1edae2-103c-4d2c-8e79-c29524a82574 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:40:01.952988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1edae2-103c-4d2c-8e79-c29524a82574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:40:01.979881 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5c1edae2-103c-4d2c-8e79-c29524a82574 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:40:01.981614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5c1edae2-103c-4d2c-8e79-c29524a82574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:40:02.068686 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1353/2724] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:40:02.091416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3ce19-64bd-4ca6-b861-2aac291ee2d8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:40:02.093632 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-69c3ce19-64bd-4ca6-b861-2aac291ee2d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:40:02.187806 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3ce19-64bd-4ca6-b861-2aac291ee2d8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:40:02.192857 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab1c9c-b6e3-4f31-a03a-eb40396d6486 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:40:02.193506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab1c9c-b6e3-4f31-a03a-eb40396d6486 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:40:02.193831 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab1c9c-b6e3-4f31-a03a-eb40396d6486 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:40:02.194118 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab1c9c-b6e3-4f31-a03a-eb40396d6486 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:40:02.210210 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab1c9c-b6e3-4f31-a03a-eb40396d6486 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:40:02.215359 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab1c9c-b6e3-4f31-a03a-eb40396d6486 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:40:02.226671 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1372/2725] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:40:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:40:02.726000 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1edae2-103c-4d2c-8e79-c29524a82574 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:40:02.727942 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5c1edae2-103c-4d2c-8e79-c29524a82574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:40:02.765541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1edae2-103c-4d2c-8e79-c29524a82574 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:40:02.775481 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a043982-e4be-4dce-b2be-11410cf56972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:40:02.777383 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a043982-e4be-4dce-b2be-11410cf56972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:40:02.777778 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a043982-e4be-4dce-b2be-11410cf56972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:40:02.778147 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a043982-e4be-4dce-b2be-11410cf56972 admin admin] PATH_INFO: `/v3/projects/ccda19fcc3404d1a801c2293193e7bb5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:40:02.779725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a043982-e4be-4dce-b2be-11410cf56972 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccda19fcc3404d1a801c2293193e7bb5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:40:02.781965 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a043982-e4be-4dce-b2be-11410cf56972 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:40:02.872286 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8a043982-e4be-4dce-b2be-11410cf56972 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccda19fcc3404d1a801c2293193e7bb5', 'request_id': 'req-8a043982-e4be-4dce-b2be-11410cf56972'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:02.883459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8a043982-e4be-4dce-b2be-11410cf56972 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccda19fcc3404d1a801c2293193e7bb5', 'request_id': 'req-8a043982-e4be-4dce-b2be-11410cf56972'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:02.889852 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8a043982-e4be-4dce-b2be-11410cf56972 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccda19fcc3404d1a801c2293193e7bb5', 'request_id': 'req-8a043982-e4be-4dce-b2be-11410cf56972'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:02.908755 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8a043982-e4be-4dce-b2be-11410cf56972 admin admin] The token cache is being invalidate because project ccda19fcc3404d1a801c2293193e7bb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:40:02.909209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8a043982-e4be-4dce-b2be-11410cf56972 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:02.915399 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-8a043982-e4be-4dce-b2be-11410cf56972 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:02.917398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1354/2726] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:40:02 2022] DELETE /identity/v3/projects/ccda19fcc3404d1a801c2293193e7bb5 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:40:12.221049 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=61,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:40:12.903421 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:40:14.341247 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab1c9c-b6e3-4f31-a03a-eb40396d6486 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:40:14.342876 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2fab1c9c-b6e3-4f31-a03a-eb40396d6486 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:40:14.429825 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab1c9c-b6e3-4f31-a03a-eb40396d6486 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:40:14.494407 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60cf4a-f751-40d8-951c-14fd12e74856 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:40:14.495059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60cf4a-f751-40d8-951c-14fd12e74856 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:40:14.495349 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60cf4a-f751-40d8-951c-14fd12e74856 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:40:14.495631 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60cf4a-f751-40d8-951c-14fd12e74856 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:40:14.497401 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fb60cf4a-f751-40d8-951c-14fd12e74856 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:40:14.509916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb60cf4a-f751-40d8-951c-14fd12e74856 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:40:14.511577 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb60cf4a-f751-40d8-951c-14fd12e74856 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:40:14.522854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1373/2727] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:40:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:40:18.897053 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a043982-e4be-4dce-b2be-11410cf56972 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:40:18.898775 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-8a043982-e4be-4dce-b2be-11410cf56972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:40:18.915334 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a043982-e4be-4dce-b2be-11410cf56972 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:40:18.920785 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f7ed9-33c3-48a0-ac83-9c4d0c2d43e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:40:18.922217 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f7ed9-33c3-48a0-ac83-9c4d0c2d43e7 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:40:18.922732 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f7ed9-33c3-48a0-ac83-9c4d0c2d43e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:40:18.922916 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f7ed9-33c3-48a0-ac83-9c4d0c2d43e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:40:18.924570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e41f7ed9-33c3-48a0-ac83-9c4d0c2d43e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:40:18.936787 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f7ed9-33c3-48a0-ac83-9c4d0c2d43e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:40:18.939079 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f7ed9-33c3-48a0-ac83-9c4d0c2d43e7 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:40:18.950230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1355/2728] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:40:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:40:20.378999 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60cf4a-f751-40d8-951c-14fd12e74856 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:40:20.380673 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fb60cf4a-f751-40d8-951c-14fd12e74856 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:40:20.441814 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60cf4a-f751-40d8-951c-14fd12e74856 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:40:20.446140 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ddb7a-4207-4fba-945b-541390fa4d4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:40:20.446799 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ddb7a-4207-4fba-945b-541390fa4d4d service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:40:20.447087 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ddb7a-4207-4fba-945b-541390fa4d4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:40:20.447376 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ddb7a-4207-4fba-945b-541390fa4d4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:40:20.449292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4a3ddb7a-4207-4fba-945b-541390fa4d4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:40:20.490301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:40:20.516657 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ddb7a-4207-4fba-945b-541390fa4d4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:40:20.518144 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ddb7a-4207-4fba-945b-541390fa4d4d service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:40:20.527868 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1374/2729] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:40:20 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:40:28.945134 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=6 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:40:30.523303 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=12 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:41:44.590315 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8b5d3-ce68-433e-b3e7-fad699b5300d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:41:44.590924 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8b5d3-ce68-433e-b3e7-fad699b5300d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:41:44.590924 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8b5d3-ce68-433e-b3e7-fad699b5300d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:41:44.617908 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-5de8b5d3-ce68-433e-b3e7-fad699b5300d None None] MFA Rules not processed for user `9531d18446eb4488bbd1511cb3412925`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:41:44.619233 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5de8b5d3-ce68-433e-b3e7-fad699b5300d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:41:44.691899 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1356/2730] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:41:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:41:50.936989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b088ca9-93b6-4db2-8365-1f5a9925d28a None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:41:50.937454 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b088ca9-93b6-4db2-8365-1f5a9925d28a None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:41:50.937554 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b088ca9-93b6-4db2-8365-1f5a9925d28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:41:50.954650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-1b088ca9-93b6-4db2-8365-1f5a9925d28a None None] MFA Rules not processed for user `4ddcf6306e704fe9ad5ed8779178ae67`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:41:50.955852 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1b088ca9-93b6-4db2-8365-1f5a9925d28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:41:51.032338 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1375/2731] 10.222.0.33 () {58 vars in 967 bytes} [Thu Jun 30 18:41:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:41:51.082641 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8b5d3-ce68-433e-b3e7-fad699b5300d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:41:51.084603 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5de8b5d3-ce68-433e-b3e7-fad699b5300d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:41:51.158670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8b5d3-ce68-433e-b3e7-fad699b5300d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:41:51.164600 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aed316-001b-433b-8310-bbd3d1e63571 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bd521b9e93f4b3b8af64d46f36c019e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:41:51.165247 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aed316-001b-433b-8310-bbd3d1e63571 service swift] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:41:51.165535 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aed316-001b-433b-8310-bbd3d1e63571 service swift] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:41:51.165845 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aed316-001b-433b-8310-bbd3d1e63571 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:41:51.173841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aed316-001b-433b-8310-bbd3d1e63571 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:41:51.175427 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aed316-001b-433b-8310-bbd3d1e63571 service swift] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:41:51.184319 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1357/2732] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 30 18:41:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 18:41:54.695376 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b088ca9-93b6-4db2-8365-1f5a9925d28a None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:41:54.697864 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1b088ca9-93b6-4db2-8365-1f5a9925d28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:41:54.857073 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b088ca9-93b6-4db2-8365-1f5a9925d28a None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:41:54.863063 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d087c2-b07d-47cd-9aaf-73ab8606d8c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49325c1733174e0c98b3475a10c9b5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:41:54.864481 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d087c2-b07d-47cd-9aaf-73ab8606d8c8 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:41:54.865869 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d087c2-b07d-47cd-9aaf-73ab8606d8c8 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:41:54.866383 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d087c2-b07d-47cd-9aaf-73ab8606d8c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:41:54.868971 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f5d087c2-b07d-47cd-9aaf-73ab8606d8c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:41:54.887473 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d087c2-b07d-47cd-9aaf-73ab8606d8c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:41:54.889820 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d087c2-b07d-47cd-9aaf-73ab8606d8c8 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:41:54.902522 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1376/2733] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:41:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:41:58.101163 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aed316-001b-433b-8310-bbd3d1e63571 service swift] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:41:58.102713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-64aed316-001b-433b-8310-bbd3d1e63571 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:41:58.170936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:41:58.201406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aed316-001b-433b-8310-bbd3d1e63571 service swift] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:41:58.206106 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ec698-7873-4b63-b0e1-8a42ec8d2d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:41:58.206915 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ec698-7873-4b63-b0e1-8a42ec8d2d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:41:58.207435 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ec698-7873-4b63-b0e1-8a42ec8d2d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:41:58.207728 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ec698-7873-4b63-b0e1-8a42ec8d2d64 admin admin] PATH_INFO: `/v3/users/784fdfd56b3d4ae4b6f68b66c4f06ea5` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:41:58.208929 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ec698-7873-4b63-b0e1-8a42ec8d2d64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=784fdfd56b3d4ae4b6f68b66c4f06ea5)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:41:58.210893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ec698-7873-4b63-b0e1-8a42ec8d2d64 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:41:58.257171 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5a7ec698-7873-4b63-b0e1-8a42ec8d2d64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '784fdfd56b3d4ae4b6f68b66c4f06ea5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:41:58.257776 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5a7ec698-7873-4b63-b0e1-8a42ec8d2d64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '784fdfd56b3d4ae4b6f68b66c4f06ea5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:41:58.269406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5a7ec698-7873-4b63-b0e1-8a42ec8d2d64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '784fdfd56b3d4ae4b6f68b66c4f06ea5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:41:58.277149 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5a7ec698-7873-4b63-b0e1-8a42ec8d2d64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '784fdfd56b3d4ae4b6f68b66c4f06ea5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:41:58.277796 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5a7ec698-7873-4b63-b0e1-8a42ec8d2d64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '784fdfd56b3d4ae4b6f68b66c4f06ea5'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:41:58.290782 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1358/2734] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:41:58 2022] DELETE /identity/v3/users/784fdfd56b3d4ae4b6f68b66c4f06ea5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:41:58.535877 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d087c2-b07d-47cd-9aaf-73ab8606d8c8 service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:41:58.547909 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d087c2-b07d-47cd-9aaf-73ab8606d8c8 service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:41:58.609036 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a9c20-0bd4-4791-97f9-95932269bdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:41:58.609935 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a9c20-0bd4-4791-97f9-95932269bdb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:41:58.610216 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a9c20-0bd4-4791-97f9-95932269bdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:41:58.610488 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a9c20-0bd4-4791-97f9-95932269bdb7 admin admin] PATH_INFO: `/v3/projects/666edd3825ba4e32ac180ed230d93e85` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:41:58.611655 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a9c20-0bd4-4791-97f9-95932269bdb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=666edd3825ba4e32ac180ed230d93e85)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:41:58.614275 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a9c20-0bd4-4791-97f9-95932269bdb7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:41:58.615729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=100 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:41:58.659560 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-319a9c20-0bd4-4791-97f9-95932269bdb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '666edd3825ba4e32ac180ed230d93e85', 'request_id': 'req-319a9c20-0bd4-4791-97f9-95932269bdb7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:41:58.666398 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-319a9c20-0bd4-4791-97f9-95932269bdb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '666edd3825ba4e32ac180ed230d93e85', 'request_id': 'req-319a9c20-0bd4-4791-97f9-95932269bdb7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:41:58.672250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-319a9c20-0bd4-4791-97f9-95932269bdb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '666edd3825ba4e32ac180ed230d93e85', 'request_id': 'req-319a9c20-0bd4-4791-97f9-95932269bdb7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:41:58.685090 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-319a9c20-0bd4-4791-97f9-95932269bdb7 admin admin] The token cache is being invalidate because project 666edd3825ba4e32ac180ed230d93e85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:41:58.685375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-319a9c20-0bd4-4791-97f9-95932269bdb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:41:58.686019 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-319a9c20-0bd4-4791-97f9-95932269bdb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:41:58.687547 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1377/2735] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:41:58 2022] DELETE /identity/v3/projects/666edd3825ba4e32ac180ed230d93e85 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:41:58.691585 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ec698-7873-4b63-b0e1-8a42ec8d2d64 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:41:58.693205 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-5a7ec698-7873-4b63-b0e1-8a42ec8d2d64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:41:58.706148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ec698-7873-4b63-b0e1-8a42ec8d2d64 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:41:58.761935 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b0149-940b-4bce-adf7-875563ee2d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:41:58.762712 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b0149-940b-4bce-adf7-875563ee2d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:41:58.762998 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b0149-940b-4bce-adf7-875563ee2d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:41:58.763275 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b0149-940b-4bce-adf7-875563ee2d28 admin admin] PATH_INFO: `/v3/users/4ae86a79a2de4ff7b9d5df3a3bfcbd14` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:41:58.764460 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b0149-940b-4bce-adf7-875563ee2d28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ae86a79a2de4ff7b9d5df3a3bfcbd14)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:41:58.766427 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b0149-940b-4bce-adf7-875563ee2d28 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:41:58.812282 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e65b0149-940b-4bce-adf7-875563ee2d28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ae86a79a2de4ff7b9d5df3a3bfcbd14'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:41:58.812943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e65b0149-940b-4bce-adf7-875563ee2d28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ae86a79a2de4ff7b9d5df3a3bfcbd14'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:41:58.818840 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:41:58.828468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e65b0149-940b-4bce-adf7-875563ee2d28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ae86a79a2de4ff7b9d5df3a3bfcbd14'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:41:58.837028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e65b0149-940b-4bce-adf7-875563ee2d28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ae86a79a2de4ff7b9d5df3a3bfcbd14'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:41:58.837857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e65b0149-940b-4bce-adf7-875563ee2d28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ae86a79a2de4ff7b9d5df3a3bfcbd14'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:41:58.852608 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1359/2736] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:41:58 2022] DELETE /identity/v3/users/4ae86a79a2de4ff7b9d5df3a3bfcbd14 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:41:59.042069 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a9c20-0bd4-4791-97f9-95932269bdb7 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:41:59.043534 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-319a9c20-0bd4-4791-97f9-95932269bdb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:41:59.059526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a9c20-0bd4-4791-97f9-95932269bdb7 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:41:59.122496 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544e86b-309e-48a1-891d-49c302076fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:41:59.123361 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544e86b-309e-48a1-891d-49c302076fd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:41:59.123647 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544e86b-309e-48a1-891d-49c302076fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:41:59.123929 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544e86b-309e-48a1-891d-49c302076fd7 admin admin] PATH_INFO: `/v3/projects/22ccfe9956dd4b62b25386e0b09abc0b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:41:59.125238 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4544e86b-309e-48a1-891d-49c302076fd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22ccfe9956dd4b62b25386e0b09abc0b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:41:59.127237 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4544e86b-309e-48a1-891d-49c302076fd7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:41:59.178397 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4544e86b-309e-48a1-891d-49c302076fd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22ccfe9956dd4b62b25386e0b09abc0b', 'request_id': 'req-4544e86b-309e-48a1-891d-49c302076fd7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:41:59.186320 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4544e86b-309e-48a1-891d-49c302076fd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22ccfe9956dd4b62b25386e0b09abc0b', 'request_id': 'req-4544e86b-309e-48a1-891d-49c302076fd7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:41:59.192000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4544e86b-309e-48a1-891d-49c302076fd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22ccfe9956dd4b62b25386e0b09abc0b', 'request_id': 'req-4544e86b-309e-48a1-891d-49c302076fd7'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:41:59.203634 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4544e86b-309e-48a1-891d-49c302076fd7 admin admin] The token cache is being invalidate because project 22ccfe9956dd4b62b25386e0b09abc0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:41:59.203930 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4544e86b-309e-48a1-891d-49c302076fd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:41:59.204559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-4544e86b-309e-48a1-891d-49c302076fd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:41:59.206138 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1378/2737] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:41:59 2022] DELETE /identity/v3/projects/22ccfe9956dd4b62b25386e0b09abc0b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:41:59.211134 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53e356-01ef-466b-9ec8-619c87d398c5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:41:59.211650 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53e356-01ef-466b-9ec8-619c87d398c5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:41:59.211781 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53e356-01ef-466b-9ec8-619c87d398c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:41:59.266870 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3c53e356-01ef-466b-9ec8-619c87d398c5 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:41:59.268009 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3c53e356-01ef-466b-9ec8-619c87d398c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:41:59.327516 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1360/2738] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:41:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:41:59.335307 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b206c-5da5-472f-b218-c44d55cd093b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:41:59.335634 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b206c-5da5-472f-b218-c44d55cd093b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:41:59.335919 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b206c-5da5-472f-b218-c44d55cd093b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:41:59.355044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-d71b206c-5da5-472f-b218-c44d55cd093b None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:41:59.356252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d71b206c-5da5-472f-b218-c44d55cd093b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:41:59.365254 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1379/2739] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:41:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:41:59.369317 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53e356-01ef-466b-9ec8-619c87d398c5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:41:59.378978 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53e356-01ef-466b-9ec8-619c87d398c5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:41:59.383649 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5501e0-bd6c-4245-ab0f-d99d3e03f0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:41:59.384302 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5501e0-bd6c-4245-ab0f-d99d3e03f0f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:41:59.384586 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5501e0-bd6c-4245-ab0f-d99d3e03f0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:41:59.384863 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5501e0-bd6c-4245-ab0f-d99d3e03f0f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:41:59.385607 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5501e0-bd6c-4245-ab0f-d99d3e03f0f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:41:59.385929 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5501e0-bd6c-4245-ab0f-d99d3e03f0f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:41:59.387562 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5501e0-bd6c-4245-ab0f-d99d3e03f0f5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:41:59.399227 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1361/2740] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:41:59.403067 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b206c-5da5-472f-b218-c44d55cd093b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:41:59.413118 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b206c-5da5-472f-b218-c44d55cd093b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:41:59.417712 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53359e09-41a1-4332-a066-fa51dbe1b84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:41:59.418352 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53359e09-41a1-4332-a066-fa51dbe1b84e admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:41:59.418642 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53359e09-41a1-4332-a066-fa51dbe1b84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:41:59.419552 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53359e09-41a1-4332-a066-fa51dbe1b84e admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:41:59.420250 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53359e09-41a1-4332-a066-fa51dbe1b84e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:41:59.422188 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53359e09-41a1-4332-a066-fa51dbe1b84e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:41:59.433111 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:41:59.450696 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1380/2741] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:41:59 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:41:59.454820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5501e0-bd6c-4245-ab0f-d99d3e03f0f5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:41:59.465327 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5501e0-bd6c-4245-ab0f-d99d3e03f0f5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:41:59.520272 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae44ca-7775-4f1b-be96-ae91b7c32a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:41:59.520941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae44ca-7775-4f1b-be96-ae91b7c32a04 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:41:59.521226 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae44ca-7775-4f1b-be96-ae91b7c32a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:41:59.521503 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae44ca-7775-4f1b-be96-ae91b7c32a04 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:41:59.522229 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae44ca-7775-4f1b-be96-ae91b7c32a04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:41:59.524151 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae44ca-7775-4f1b-be96-ae91b7c32a04 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:41:59.544689 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1362/2742] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:41:59 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:41:59.548754 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53359e09-41a1-4332-a066-fa51dbe1b84e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:41:59.561649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53359e09-41a1-4332-a066-fa51dbe1b84e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:41:59.567970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e81f2-254a-4eb5-9ec0-169f77db4cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:41:59.568638 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e81f2-254a-4eb5-9ec0-169f77db4cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:41:59.568918 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e81f2-254a-4eb5-9ec0-169f77db4cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:41:59.569194 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e81f2-254a-4eb5-9ec0-169f77db4cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:41:59.569954 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e81f2-254a-4eb5-9ec0-169f77db4cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:41:59.571701 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e81f2-254a-4eb5-9ec0-169f77db4cd5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:41:59.585397 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1381/2743] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:41:59.588920 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae44ca-7775-4f1b-be96-ae91b7c32a04 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:41:59.600681 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae44ca-7775-4f1b-be96-ae91b7c32a04 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:41:59.605460 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e42163-4a38-416d-a8ab-3e10968b4fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:41:59.606154 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e42163-4a38-416d-a8ab-3e10968b4fdd admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:41:59.606439 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e42163-4a38-416d-a8ab-3e10968b4fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:41:59.606723 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e42163-4a38-416d-a8ab-3e10968b4fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:41:59.607445 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e42163-4a38-416d-a8ab-3e10968b4fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:41:59.609046 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e42163-4a38-416d-a8ab-3e10968b4fdd admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:41:59.620523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1363/2744] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:41:59.624752 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e81f2-254a-4eb5-9ec0-169f77db4cd5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:41:59.635323 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e81f2-254a-4eb5-9ec0-169f77db4cd5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:41:59.640416 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101a9a7-e189-4cd4-8daf-1622c2fdba6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:41:59.641346 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101a9a7-e189-4cd4-8daf-1622c2fdba6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:41:59.641702 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101a9a7-e189-4cd4-8daf-1622c2fdba6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:41:59.641952 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101a9a7-e189-4cd4-8daf-1622c2fdba6f admin admin] PATH_INFO: `/v3/projects/8bc355242c9d490d91fbcbd7d08f8b97/users/f7f4e45a0882410088464b550a4b06c1/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:41:59.661482 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d101a9a7-e189-4cd4-8daf-1622c2fdba6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bc355242c9d490d91fbcbd7d08f8b97, user_id=f7f4e45a0882410088464b550a4b06c1, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:41:59.664301 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d101a9a7-e189-4cd4-8daf-1622c2fdba6f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:41:59.678300 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1382/2745] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:41:59 2022] PUT /identity/v3/projects/8bc355242c9d490d91fbcbd7d08f8b97/users/f7f4e45a0882410088464b550a4b06c1/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:41:59.683595 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae01991-8f92-402b-9bc2-1b463c3eb1e9 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:41:59.683893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae01991-8f92-402b-9bc2-1b463c3eb1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:41:59.684179 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae01991-8f92-402b-9bc2-1b463c3eb1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:41:59.696098 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:41:59.704127 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-aae01991-8f92-402b-9bc2-1b463c3eb1e9 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:41:59.705900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-aae01991-8f92-402b-9bc2-1b463c3eb1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:41:59.766855 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1364/2746] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:41:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:41:59.777046 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101a9a7-e189-4cd4-8daf-1622c2fdba6f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:41:59.778759 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d101a9a7-e189-4cd4-8daf-1622c2fdba6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:41:59.860941 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101a9a7-e189-4cd4-8daf-1622c2fdba6f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:41:59.864883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610dd61-45ba-45ba-94ce-df838a730409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:41:59.865527 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610dd61-45ba-45ba-94ce-df838a730409 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:41:59.865967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610dd61-45ba-45ba-94ce-df838a730409 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:41:59.866251 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610dd61-45ba-45ba-94ce-df838a730409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:41:59.875438 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610dd61-45ba-45ba-94ce-df838a730409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:41:59.876921 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610dd61-45ba-45ba-94ce-df838a730409 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:41:59.887456 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1383/2747] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:41:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:42:04.231278 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff6546-483a-41d0-9731-033de4585439 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:42:04.231816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff6546-483a-41d0-9731-033de4585439 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:42:04.231919 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff6546-483a-41d0-9731-033de4585439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:42:04.260669 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-90ff6546-483a-41d0-9731-033de4585439 None None] MFA Rules not processed for user `f7f4e45a0882410088464b550a4b06c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:42:04.261932 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-90ff6546-483a-41d0-9731-033de4585439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:42:04.331219 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1365/2748] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:42:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:42:04.336186 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610dd61-45ba-45ba-94ce-df838a730409 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:42:04.346229 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610dd61-45ba-45ba-94ce-df838a730409 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:42:04.351802 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42b419-b03e-4dc0-93b7-8f9c5c331da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:42:04.352547 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42b419-b03e-4dc0-93b7-8f9c5c331da9 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:42:04.352911 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42b419-b03e-4dc0-93b7-8f9c5c331da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:42:04.353257 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42b419-b03e-4dc0-93b7-8f9c5c331da9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:42:04.354078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42b419-b03e-4dc0-93b7-8f9c5c331da9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:42:04.355978 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42b419-b03e-4dc0-93b7-8f9c5c331da9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:42:04.386164 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1384/2749] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:42:04 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:42:04.390124 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff6546-483a-41d0-9731-033de4585439 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:42:04.401856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff6546-483a-41d0-9731-033de4585439 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:42:04.458648 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2799d31-9655-446b-b4a9-b4240e9abd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:42:04.459310 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2799d31-9655-446b-b4a9-b4240e9abd34 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:42:04.459596 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2799d31-9655-446b-b4a9-b4240e9abd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:42:04.459894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2799d31-9655-446b-b4a9-b4240e9abd34 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:42:04.460585 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2799d31-9655-446b-b4a9-b4240e9abd34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:42:04.462524 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2799d31-9655-446b-b4a9-b4240e9abd34 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:42:04.481973 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1366/2750] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:42:04 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:42:04.485981 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42b419-b03e-4dc0-93b7-8f9c5c331da9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:42:04.496647 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42b419-b03e-4dc0-93b7-8f9c5c331da9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:42:04.501449 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9d3e2-2e5f-4302-943f-d3df57737635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:42:04.502125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9d3e2-2e5f-4302-943f-d3df57737635 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:42:04.502410 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9d3e2-2e5f-4302-943f-d3df57737635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:42:04.502691 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9d3e2-2e5f-4302-943f-d3df57737635 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:42:04.504141 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9d3e2-2e5f-4302-943f-d3df57737635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:42:04.505937 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9d3e2-2e5f-4302-943f-d3df57737635 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:42:04.515787 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1385/2751] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:42:04.520382 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2799d31-9655-446b-b4a9-b4240e9abd34 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:42:04.530755 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2799d31-9655-446b-b4a9-b4240e9abd34 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:42:04.535564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4cfa6-cad4-4592-a187-92809c3b1eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:42:04.536437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4cfa6-cad4-4592-a187-92809c3b1eff admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:42:04.536725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4cfa6-cad4-4592-a187-92809c3b1eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:42:04.537005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4cfa6-cad4-4592-a187-92809c3b1eff admin admin] PATH_INFO: `/v3/projects/5a6986baffaf4f3a842ea1fe2a974741/users/992acfd6168845a7817a7aa15fdcb1e3/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:42:04.552224 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4cfa6-cad4-4592-a187-92809c3b1eff admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a6986baffaf4f3a842ea1fe2a974741, user_id=992acfd6168845a7817a7aa15fdcb1e3, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:42:04.554934 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4cfa6-cad4-4592-a187-92809c3b1eff admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:42:04.571165 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1367/2752] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:42:04 2022] PUT /identity/v3/projects/5a6986baffaf4f3a842ea1fe2a974741/users/992acfd6168845a7817a7aa15fdcb1e3/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:42:04.575264 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9d3e2-2e5f-4302-943f-d3df57737635 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:42:04.586406 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9d3e2-2e5f-4302-943f-d3df57737635 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:42:04.643665 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830828ab-9206-4614-be8e-221c0449b094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:42:04.645107 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830828ab-9206-4614-be8e-221c0449b094 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:42:04.645394 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830828ab-9206-4614-be8e-221c0449b094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:42:04.645715 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830828ab-9206-4614-be8e-221c0449b094 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:42:04.646470 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830828ab-9206-4614-be8e-221c0449b094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:42:04.648078 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830828ab-9206-4614-be8e-221c0449b094 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:42:04.657863 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1386/2753] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:42:04.662195 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4cfa6-cad4-4592-a187-92809c3b1eff admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:42:04.667659 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:42:04.674166 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4cfa6-cad4-4592-a187-92809c3b1eff admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:42:04.679694 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c343a-3fdf-43a8-a475-f563b58914c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:42:04.680569 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c343a-3fdf-43a8-a475-f563b58914c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:42:04.680854 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c343a-3fdf-43a8-a475-f563b58914c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:42:04.681405 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c343a-3fdf-43a8-a475-f563b58914c4 admin admin] PATH_INFO: `/v3/projects/5a6986baffaf4f3a842ea1fe2a974741/users/992acfd6168845a7817a7aa15fdcb1e3/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:42:04.683532 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c343a-3fdf-43a8-a475-f563b58914c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a6986baffaf4f3a842ea1fe2a974741, user_id=992acfd6168845a7817a7aa15fdcb1e3, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:42:04.686233 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c343a-3fdf-43a8-a475-f563b58914c4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:42:04.696912 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1368/2754] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:42:04 2022] PUT /identity/v3/projects/5a6986baffaf4f3a842ea1fe2a974741/users/992acfd6168845a7817a7aa15fdcb1e3/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:42:08.856803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83dff4-cde4-4642-9a19-76c948f94ebe None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:42:08.857346 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83dff4-cde4-4642-9a19-76c948f94ebe None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:42:08.857514 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83dff4-cde4-4642-9a19-76c948f94ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:42:08.865294 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:42:08.890923 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7a83dff4-cde4-4642-9a19-76c948f94ebe None None] MFA Rules not processed for user `992acfd6168845a7817a7aa15fdcb1e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:42:08.892094 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7a83dff4-cde4-4642-9a19-76c948f94ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:42:08.966516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1387/2755] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:42:08 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:42:08.980948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c343a-3fdf-43a8-a475-f563b58914c4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:42:08.982765 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-319c343a-3fdf-43a8-a475-f563b58914c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:42:09.078788 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c343a-3fdf-43a8-a475-f563b58914c4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:42:09.084981 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc16df63-64d7-47a4-b58e-36632a44b4e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:42:09.085661 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16df63-64d7-47a4-b58e-36632a44b4e8 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:42:09.085951 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16df63-64d7-47a4-b58e-36632a44b4e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:42:09.086242 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16df63-64d7-47a4-b58e-36632a44b4e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:42:09.163885 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc16df63-64d7-47a4-b58e-36632a44b4e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:42:09.165505 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc16df63-64d7-47a4-b58e-36632a44b4e8 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:42:09.176041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1369/2756] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:42:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:42:10.259727 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475300a-43f1-4200-9598-260ebfc00829 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:42:10.260174 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475300a-43f1-4200-9598-260ebfc00829 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:42:10.260307 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475300a-43f1-4200-9598-260ebfc00829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:42:10.278479 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-7475300a-43f1-4200-9598-260ebfc00829 None None] MFA Rules not processed for user `f7f4e45a0882410088464b550a4b06c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:42:10.279803 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7475300a-43f1-4200-9598-260ebfc00829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:42:10.291513 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1388/2757] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:42:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:42:19.170636 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=60,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:42:20.272692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=36 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:42:33.106811 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc16df63-64d7-47a4-b58e-36632a44b4e8 service nova] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:42:33.108212 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-dc16df63-64d7-47a4-b58e-36632a44b4e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:42:33.165034 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc16df63-64d7-47a4-b58e-36632a44b4e8 service nova] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:42:33.168550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d801d2f-711f-4692-a573-40dcedb0ed18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:42:33.169180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d801d2f-711f-4692-a573-40dcedb0ed18 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:42:33.169459 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d801d2f-711f-4692-a573-40dcedb0ed18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:42:33.169770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d801d2f-711f-4692-a573-40dcedb0ed18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:42:33.171499 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1d801d2f-711f-4692-a573-40dcedb0ed18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:42:33.181647 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d801d2f-711f-4692-a573-40dcedb0ed18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:42:33.183066 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d801d2f-711f-4692-a573-40dcedb0ed18 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:42:33.191551 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1370/2758] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:42:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:42:34.100829 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475300a-43f1-4200-9598-260ebfc00829 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:42:34.102292 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-7475300a-43f1-4200-9598-260ebfc00829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:42:34.170506 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475300a-43f1-4200-9598-260ebfc00829 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:42:34.175610 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b5987-250b-4ad4-9a76-c1a6e20a4217 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49325c1733174e0c98b3475a10c9b5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:42:34.176254 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b5987-250b-4ad4-9a76-c1a6e20a4217 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:42:34.176542 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b5987-250b-4ad4-9a76-c1a6e20a4217 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:42:34.176824 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b5987-250b-4ad4-9a76-c1a6e20a4217 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:42:34.178873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-c48b5987-250b-4ad4-9a76-c1a6e20a4217 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:42:34.190148 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b5987-250b-4ad4-9a76-c1a6e20a4217 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:42:34.191719 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b5987-250b-4ad4-9a76-c1a6e20a4217 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:42:34.201455 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1389/2759] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:42:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:42:43.187345 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=26 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:42:44.197267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=32 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:42:50.186331 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d801d2f-711f-4692-a573-40dcedb0ed18 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:42:50.195843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d801d2f-711f-4692-a573-40dcedb0ed18 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:42:50.199450 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f0c5e-d2bd-4f06-b4b6-f38cf8fc4098 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:42:50.200096 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f0c5e-d2bd-4f06-b4b6-f38cf8fc4098 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:42:50.200409 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f0c5e-d2bd-4f06-b4b6-f38cf8fc4098 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:42:50.200703 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f0c5e-d2bd-4f06-b4b6-f38cf8fc4098 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:42:50.203355 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-650f0c5e-d2bd-4f06-b4b6-f38cf8fc4098 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:42:50.270481 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f0c5e-d2bd-4f06-b4b6-f38cf8fc4098 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:42:50.271939 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f0c5e-d2bd-4f06-b4b6-f38cf8fc4098 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:42:50.282322 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1371/2760] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:42:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:42:51.659011 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b5987-250b-4ad4-9a76-c1a6e20a4217 service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:42:51.668519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b5987-250b-4ad4-9a76-c1a6e20a4217 service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:42:51.673006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed6f62-fd28-4791-8d2a-f41313f5406b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:42:51.673673 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed6f62-fd28-4791-8d2a-f41313f5406b service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:42:51.673961 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed6f62-fd28-4791-8d2a-f41313f5406b service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:42:51.674242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed6f62-fd28-4791-8d2a-f41313f5406b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:42:51.684655 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ed6f62-fd28-4791-8d2a-f41313f5406b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:42:51.686550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ed6f62-fd28-4791-8d2a-f41313f5406b service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:42:51.698267 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1390/2761] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:42:51 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:42:57.238548 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f0c5e-d2bd-4f06-b4b6-f38cf8fc4098 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:42:57.240193 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-650f0c5e-d2bd-4f06-b4b6-f38cf8fc4098 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:42:57.253974 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f0c5e-d2bd-4f06-b4b6-f38cf8fc4098 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:42:57.259350 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af0e52-be77-4f70-8aee-4c67fe06d578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:42:57.260256 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af0e52-be77-4f70-8aee-4c67fe06d578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:42:57.260536 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af0e52-be77-4f70-8aee-4c67fe06d578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:42:57.260816 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af0e52-be77-4f70-8aee-4c67fe06d578 admin admin] PATH_INFO: `/v3/users/31c0594d0be84154a601d625bca699cd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:42:57.262044 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af0e52-be77-4f70-8aee-4c67fe06d578 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31c0594d0be84154a601d625bca699cd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:42:57.264707 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af0e52-be77-4f70-8aee-4c67fe06d578 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:42:57.315120 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b7af0e52-be77-4f70-8aee-4c67fe06d578 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31c0594d0be84154a601d625bca699cd'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:42:57.316163 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b7af0e52-be77-4f70-8aee-4c67fe06d578 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31c0594d0be84154a601d625bca699cd'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:42:57.330630 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b7af0e52-be77-4f70-8aee-4c67fe06d578 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31c0594d0be84154a601d625bca699cd'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:42:57.338575 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b7af0e52-be77-4f70-8aee-4c67fe06d578 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31c0594d0be84154a601d625bca699cd'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:42:57.339273 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b7af0e52-be77-4f70-8aee-4c67fe06d578 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31c0594d0be84154a601d625bca699cd'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:42:57.352494 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1372/2762] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:42:57 2022] DELETE /identity/v3/users/31c0594d0be84154a601d625bca699cd => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:42:57.531953 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed6f62-fd28-4791-8d2a-f41313f5406b service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:42:57.543266 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed6f62-fd28-4791-8d2a-f41313f5406b service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:42:57.603177 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad9722-8bec-42af-980a-131f83e32fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:42:57.604042 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad9722-8bec-42af-980a-131f83e32fab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:42:57.604326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad9722-8bec-42af-980a-131f83e32fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:42:57.604599 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad9722-8bec-42af-980a-131f83e32fab admin admin] PATH_INFO: `/v3/projects/64655c9e82bb4515a976f41ac93b242f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:42:57.605791 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad9722-8bec-42af-980a-131f83e32fab admin admin] RBAC: Authorizing `identity:delete_project(project_id=64655c9e82bb4515a976f41ac93b242f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:42:57.607770 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad9722-8bec-42af-980a-131f83e32fab admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:42:57.653358 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-acad9722-8bec-42af-980a-131f83e32fab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64655c9e82bb4515a976f41ac93b242f', 'request_id': 'req-acad9722-8bec-42af-980a-131f83e32fab'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:42:57.659897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-acad9722-8bec-42af-980a-131f83e32fab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64655c9e82bb4515a976f41ac93b242f', 'request_id': 'req-acad9722-8bec-42af-980a-131f83e32fab'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:42:57.665871 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-acad9722-8bec-42af-980a-131f83e32fab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64655c9e82bb4515a976f41ac93b242f', 'request_id': 'req-acad9722-8bec-42af-980a-131f83e32fab'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:42:57.680446 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-acad9722-8bec-42af-980a-131f83e32fab admin admin] The token cache is being invalidate because project 64655c9e82bb4515a976f41ac93b242f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:42:57.680744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-acad9722-8bec-42af-980a-131f83e32fab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:42:57.681372 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-acad9722-8bec-42af-980a-131f83e32fab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:42:57.682957 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1391/2763] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:42:57 2022] DELETE /identity/v3/projects/64655c9e82bb4515a976f41ac93b242f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:42:57.687882 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af0e52-be77-4f70-8aee-4c67fe06d578 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:42:57.689371 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-b7af0e52-be77-4f70-8aee-4c67fe06d578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:42:57.704099 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af0e52-be77-4f70-8aee-4c67fe06d578 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:42:57.769827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a49d67-d002-4ef0-a8ce-cc2cce520ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:42:57.770604 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a49d67-d002-4ef0-a8ce-cc2cce520ade admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:42:57.770892 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a49d67-d002-4ef0-a8ce-cc2cce520ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:42:57.771171 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a49d67-d002-4ef0-a8ce-cc2cce520ade admin admin] PATH_INFO: `/v3/users/4b80bd5d4c674b6c98cb91b1de7bf01b` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:42:57.772361 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a49d67-d002-4ef0-a8ce-cc2cce520ade admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b80bd5d4c674b6c98cb91b1de7bf01b)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:42:57.774336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a49d67-d002-4ef0-a8ce-cc2cce520ade admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:42:57.779887 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:42:57.822305 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e8a49d67-d002-4ef0-a8ce-cc2cce520ade admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b80bd5d4c674b6c98cb91b1de7bf01b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:42:57.822970 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e8a49d67-d002-4ef0-a8ce-cc2cce520ade admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b80bd5d4c674b6c98cb91b1de7bf01b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:42:57.835146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e8a49d67-d002-4ef0-a8ce-cc2cce520ade admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b80bd5d4c674b6c98cb91b1de7bf01b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:42:57.842344 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e8a49d67-d002-4ef0-a8ce-cc2cce520ade admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b80bd5d4c674b6c98cb91b1de7bf01b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:42:57.842955 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e8a49d67-d002-4ef0-a8ce-cc2cce520ade admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b80bd5d4c674b6c98cb91b1de7bf01b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:42:57.854951 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1373/2764] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:42:57 2022] DELETE /identity/v3/users/4b80bd5d4c674b6c98cb91b1de7bf01b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:42:58.035258 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad9722-8bec-42af-980a-131f83e32fab admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:42:58.036692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-acad9722-8bec-42af-980a-131f83e32fab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:42:58.050316 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad9722-8bec-42af-980a-131f83e32fab admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:42:58.113774 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab265fe4-6b9a-4bac-8238-7d4b655cb01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:42:58.114629 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab265fe4-6b9a-4bac-8238-7d4b655cb01f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:42:58.114917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab265fe4-6b9a-4bac-8238-7d4b655cb01f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:42:58.115200 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab265fe4-6b9a-4bac-8238-7d4b655cb01f admin admin] PATH_INFO: `/v3/projects/7571f5ecf2c34afe98111f44580632f8` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:42:58.116376 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab265fe4-6b9a-4bac-8238-7d4b655cb01f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7571f5ecf2c34afe98111f44580632f8)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:42:58.118340 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab265fe4-6b9a-4bac-8238-7d4b655cb01f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:42:58.140192 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:42:58.167581 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ab265fe4-6b9a-4bac-8238-7d4b655cb01f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7571f5ecf2c34afe98111f44580632f8', 'request_id': 'req-ab265fe4-6b9a-4bac-8238-7d4b655cb01f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:42:58.174691 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ab265fe4-6b9a-4bac-8238-7d4b655cb01f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7571f5ecf2c34afe98111f44580632f8', 'request_id': 'req-ab265fe4-6b9a-4bac-8238-7d4b655cb01f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:42:58.180525 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ab265fe4-6b9a-4bac-8238-7d4b655cb01f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7571f5ecf2c34afe98111f44580632f8', 'request_id': 'req-ab265fe4-6b9a-4bac-8238-7d4b655cb01f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:42:58.193131 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ab265fe4-6b9a-4bac-8238-7d4b655cb01f admin admin] The token cache is being invalidate because project 7571f5ecf2c34afe98111f44580632f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:42:58.193419 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ab265fe4-6b9a-4bac-8238-7d4b655cb01f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:42:58.194071 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ab265fe4-6b9a-4bac-8238-7d4b655cb01f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:42:58.195602 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1392/2765] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:42:58 2022] DELETE /identity/v3/projects/7571f5ecf2c34afe98111f44580632f8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:43:07.851605 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:43:08.190376 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:43:29.168370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f26ab-edd3-46dc-a816-ca15a714bb50 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:43:29.168940 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f26ab-edd3-46dc-a816-ca15a714bb50 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:43:29.169040 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f26ab-edd3-46dc-a816-ca15a714bb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:43:29.186234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-fd4f26ab-edd3-46dc-a816-ca15a714bb50 None None] MFA Rules not processed for user `9531d18446eb4488bbd1511cb3412925`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:43:29.187452 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fd4f26ab-edd3-46dc-a816-ca15a714bb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:43:29.242032 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1374/2766] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:43:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:43:29.278271 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab265fe4-6b9a-4bac-8238-7d4b655cb01f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:43:29.279703 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ab265fe4-6b9a-4bac-8238-7d4b655cb01f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:43:29.336074 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab265fe4-6b9a-4bac-8238-7d4b655cb01f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:43:29.340331 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8e31b-6aec-453e-ae99-3ebe8af75f68 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37a9fea702cd405ea9fc14ec2c689648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:43:29.340972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8e31b-6aec-453e-ae99-3ebe8af75f68 service glance] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:43:29.341260 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8e31b-6aec-453e-ae99-3ebe8af75f68 service glance] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:43:29.341542 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8e31b-6aec-453e-ae99-3ebe8af75f68 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:43:29.343325 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e7e8e31b-6aec-453e-ae99-3ebe8af75f68 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:43:29.431252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8e31b-6aec-453e-ae99-3ebe8af75f68 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:43:29.433758 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8e31b-6aec-453e-ae99-3ebe8af75f68 service glance] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:43:29.453553 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1393/2767] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:43:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:43:30.295664 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f26ab-edd3-46dc-a816-ca15a714bb50 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:43:30.297156 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fd4f26ab-edd3-46dc-a816-ca15a714bb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:43:30.359224 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f26ab-edd3-46dc-a816-ca15a714bb50 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:43:30.362782 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea789655-b50e-4690-8030-64ae150b0516 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:43:30.363422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea789655-b50e-4690-8030-64ae150b0516 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:43:30.363705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea789655-b50e-4690-8030-64ae150b0516 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:43:30.363985 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea789655-b50e-4690-8030-64ae150b0516 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:43:30.365814 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ea789655-b50e-4690-8030-64ae150b0516 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:43:30.380633 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea789655-b50e-4690-8030-64ae150b0516 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:43:30.382140 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea789655-b50e-4690-8030-64ae150b0516 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:43:30.394087 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1375/2768] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:43:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:43:39.450957 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=66 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:43:40.388518 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=51 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:44:40.100154 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8e31b-6aec-453e-ae99-3ebe8af75f68 service glance] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:44:40.110158 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8e31b-6aec-453e-ae99-3ebe8af75f68 service glance] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:44:40.113781 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f58338-6402-4e9d-94e9-a10618f25e9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:44:40.114424 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f58338-6402-4e9d-94e9-a10618f25e9d service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:44:40.114712 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f58338-6402-4e9d-94e9-a10618f25e9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:44:40.114995 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f58338-6402-4e9d-94e9-a10618f25e9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:44:40.116764 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-98f58338-6402-4e9d-94e9-a10618f25e9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:44:40.128013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f58338-6402-4e9d-94e9-a10618f25e9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:44:40.129436 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f58338-6402-4e9d-94e9-a10618f25e9d service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:44:40.137403 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1394/2769] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:44:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:44:50.133927 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=6 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:44:59.664919 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea789655-b50e-4690-8030-64ae150b0516 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:44:59.666948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ea789655-b50e-4690-8030-64ae150b0516 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:44:59.747555 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea789655-b50e-4690-8030-64ae150b0516 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:44:59.751219 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5089c-f4eb-4626-813b-cba3004691e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:44:59.752002 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5089c-f4eb-4626-813b-cba3004691e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:44:59.752290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5089c-f4eb-4626-813b-cba3004691e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:44:59.752575 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5089c-f4eb-4626-813b-cba3004691e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:44:59.754419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-15d5089c-f4eb-4626-813b-cba3004691e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:44:59.766529 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d5089c-f4eb-4626-813b-cba3004691e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:44:59.768008 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d5089c-f4eb-4626-813b-cba3004691e2 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:44:59.777291 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1376/2770] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:44:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:45:09.773034 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=35 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:45:12.512443 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0afb5-fb57-4e9b-8561-dea61bfee9b8 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:45:12.512951 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0afb5-fb57-4e9b-8561-dea61bfee9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:45:12.513074 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0afb5-fb57-4e9b-8561-dea61bfee9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:45:12.529771 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b2d0afb5-fb57-4e9b-8561-dea61bfee9b8 None None] MFA Rules not processed for user `9531d18446eb4488bbd1511cb3412925`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:45:12.530972 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b2d0afb5-fb57-4e9b-8561-dea61bfee9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:45:12.537812 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1395/2771] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:45:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 26 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:45:15.232812 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5089c-f4eb-4626-813b-cba3004691e2 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:45:15.234210 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-15d5089c-f4eb-4626-813b-cba3004691e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:45:15.301739 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5089c-f4eb-4626-813b-cba3004691e2 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:45:15.307016 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751c233-3462-4cc0-a064-d4cf826ac9e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49325c1733174e0c98b3475a10c9b5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:45:15.307651 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751c233-3462-4cc0-a064-d4cf826ac9e2 service placement] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:45:15.307932 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751c233-3462-4cc0-a064-d4cf826ac9e2 service placement] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:45:15.308208 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751c233-3462-4cc0-a064-d4cf826ac9e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:45:15.309991 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1751c233-3462-4cc0-a064-d4cf826ac9e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:45:15.322434 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1751c233-3462-4cc0-a064-d4cf826ac9e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:45:15.324101 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1751c233-3462-4cc0-a064-d4cf826ac9e2 service placement] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:45:15.334297 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1377/2772] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:45:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:45:19.247151 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0afb5-fb57-4e9b-8561-dea61bfee9b8 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:45:19.249214 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b2d0afb5-fb57-4e9b-8561-dea61bfee9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:45:19.276704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0afb5-fb57-4e9b-8561-dea61bfee9b8 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:45:19.282526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec078c-8df3-4111-936d-adf45b1bb533 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:45:19.283235 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec078c-8df3-4111-936d-adf45b1bb533 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:45:19.283609 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec078c-8df3-4111-936d-adf45b1bb533 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:45:19.283979 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec078c-8df3-4111-936d-adf45b1bb533 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:45:19.285910 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2cec078c-8df3-4111-936d-adf45b1bb533 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:45:19.300079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec078c-8df3-4111-936d-adf45b1bb533 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:45:19.301729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec078c-8df3-4111-936d-adf45b1bb533 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:45:19.313093 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1396/2773] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:45:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:45:21.717094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751c233-3462-4cc0-a064-d4cf826ac9e2 service placement] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:45:21.727443 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751c233-3462-4cc0-a064-d4cf826ac9e2 service placement] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:45:21.730980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d6165-c346-4d67-acf2-4d166b0be845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:45:21.731605 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d6165-c346-4d67-acf2-4d166b0be845 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:45:21.731894 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d6165-c346-4d67-acf2-4d166b0be845 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:45:21.732170 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d6165-c346-4d67-acf2-4d166b0be845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:45:21.733982 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-852d6165-c346-4d67-acf2-4d166b0be845 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:45:21.743591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d6165-c346-4d67-acf2-4d166b0be845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:45:21.745009 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d6165-c346-4d67-acf2-4d166b0be845 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:45:21.752924 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1378/2774] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:45:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:45:21.847644 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec078c-8df3-4111-936d-adf45b1bb533 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:45:21.849176 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2cec078c-8df3-4111-936d-adf45b1bb533 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:45:21.860819 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec078c-8df3-4111-936d-adf45b1bb533 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:45:21.864449 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e849e6-bfa8-4153-908a-2aed4f3818f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:45:21.865114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e849e6-bfa8-4153-908a-2aed4f3818f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:45:21.865400 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e849e6-bfa8-4153-908a-2aed4f3818f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:45:21.865715 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e849e6-bfa8-4153-908a-2aed4f3818f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:45:21.867537 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b3e849e6-bfa8-4153-908a-2aed4f3818f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:45:21.878013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e849e6-bfa8-4153-908a-2aed4f3818f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:45:21.879466 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e849e6-bfa8-4153-908a-2aed4f3818f1 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:45:21.888713 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1397/2775] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:45:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:45:21.965438 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d6165-c346-4d67-acf2-4d166b0be845 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:45:21.974060 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d6165-c346-4d67-acf2-4d166b0be845 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:45:21.977912 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ddef5b-33dc-450c-ab5d-3bbb53df105c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:45:21.978549 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddef5b-33dc-450c-ab5d-3bbb53df105c service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:45:21.978834 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddef5b-33dc-450c-ab5d-3bbb53df105c service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:45:21.979113 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddef5b-33dc-450c-ab5d-3bbb53df105c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:45:21.981007 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-07ddef5b-33dc-450c-ab5d-3bbb53df105c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:45:21.996803 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ddef5b-33dc-450c-ab5d-3bbb53df105c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:45:21.998286 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ddef5b-33dc-450c-ab5d-3bbb53df105c service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:45:22.012943 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1379/2776] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:45:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:45:23.945539 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e849e6-bfa8-4153-908a-2aed4f3818f1 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:45:23.964102 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e849e6-bfa8-4153-908a-2aed4f3818f1 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:45:23.972389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16468f-92b6-4568-aa6d-223b518328ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:45:23.973302 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16468f-92b6-4568-aa6d-223b518328ba service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:45:23.973522 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16468f-92b6-4568-aa6d-223b518328ba service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:45:23.973848 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16468f-92b6-4568-aa6d-223b518328ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:45:23.977124 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2d16468f-92b6-4568-aa6d-223b518328ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:45:23.990888 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16468f-92b6-4568-aa6d-223b518328ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:45:23.993165 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16468f-92b6-4568-aa6d-223b518328ba service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:45:24.004326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1398/2777] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:45:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:45:32.006288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=44 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:45:33.999578 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=21 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:47:03.108819 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ddef5b-33dc-450c-ab5d-3bbb53df105c service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:03.119447 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ddef5b-33dc-450c-ab5d-3bbb53df105c service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:03.123812 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863d9cb-d119-4979-8a93-d3988268f156 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:03.124448 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863d9cb-d119-4979-8a93-d3988268f156 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:03.124730 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863d9cb-d119-4979-8a93-d3988268f156 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:03.125006 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863d9cb-d119-4979-8a93-d3988268f156 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:03.126790 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3863d9cb-d119-4979-8a93-d3988268f156 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:03.190276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863d9cb-d119-4979-8a93-d3988268f156 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:03.191716 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863d9cb-d119-4979-8a93-d3988268f156 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:03.202101 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1380/2778] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:47:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:47:06.301206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16468f-92b6-4568-aa6d-223b518328ba service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:06.317097 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16468f-92b6-4568-aa6d-223b518328ba service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:06.322128 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bff8ae-be62-4b8f-a2c8-c785dddb8b58 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49325c1733174e0c98b3475a10c9b5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:06.323331 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bff8ae-be62-4b8f-a2c8-c785dddb8b58 service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:06.324156 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bff8ae-be62-4b8f-a2c8-c785dddb8b58 service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:06.324438 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bff8ae-be62-4b8f-a2c8-c785dddb8b58 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:06.326375 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-52bff8ae-be62-4b8f-a2c8-c785dddb8b58 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:06.338406 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bff8ae-be62-4b8f-a2c8-c785dddb8b58 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:06.340727 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bff8ae-be62-4b8f-a2c8-c785dddb8b58 service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:06.351516 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1399/2779] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:47:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:47:06.356685 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863d9cb-d119-4979-8a93-d3988268f156 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:06.358184 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3863d9cb-d119-4979-8a93-d3988268f156 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:06.371631 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863d9cb-d119-4979-8a93-d3988268f156 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:06.376695 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e9689-d04f-4e9c-8d92-275d2d8cbbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:06.377694 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e9689-d04f-4e9c-8d92-275d2d8cbbe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:06.378043 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e9689-d04f-4e9c-8d92-275d2d8cbbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:06.378382 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e9689-d04f-4e9c-8d92-275d2d8cbbe0 admin admin] PATH_INFO: `/v3/users/9531d18446eb4488bbd1511cb3412925` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:06.379655 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e9689-d04f-4e9c-8d92-275d2d8cbbe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9531d18446eb4488bbd1511cb3412925)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:06.381569 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e9689-d04f-4e9c-8d92-275d2d8cbbe0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:06.433725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c13e9689-d04f-4e9c-8d92-275d2d8cbbe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9531d18446eb4488bbd1511cb3412925'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:06.434494 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c13e9689-d04f-4e9c-8d92-275d2d8cbbe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9531d18446eb4488bbd1511cb3412925'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:06.446324 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c13e9689-d04f-4e9c-8d92-275d2d8cbbe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9531d18446eb4488bbd1511cb3412925'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:06.454271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c13e9689-d04f-4e9c-8d92-275d2d8cbbe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9531d18446eb4488bbd1511cb3412925'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:06.454872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-c13e9689-d04f-4e9c-8d92-275d2d8cbbe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9531d18446eb4488bbd1511cb3412925'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:06.467878 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1381/2780] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:47:06 2022] DELETE /identity/v3/users/9531d18446eb4488bbd1511cb3412925 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:47:06.644647 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bff8ae-be62-4b8f-a2c8-c785dddb8b58 service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:06.656966 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bff8ae-be62-4b8f-a2c8-c785dddb8b58 service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:06.715855 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e53ce6-3479-451d-b5d5-582e97925dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:06.716718 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e53ce6-3479-451d-b5d5-582e97925dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:06.717007 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e53ce6-3479-451d-b5d5-582e97925dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:06.717290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e53ce6-3479-451d-b5d5-582e97925dcb admin admin] PATH_INFO: `/v3/projects/b8227be3d9a044ed888b2f54abd9463d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:06.718535 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e53ce6-3479-451d-b5d5-582e97925dcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8227be3d9a044ed888b2f54abd9463d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:06.720460 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e53ce6-3479-451d-b5d5-582e97925dcb admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:06.766035 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-18e53ce6-3479-451d-b5d5-582e97925dcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8227be3d9a044ed888b2f54abd9463d', 'request_id': 'req-18e53ce6-3479-451d-b5d5-582e97925dcb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:06.772028 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-18e53ce6-3479-451d-b5d5-582e97925dcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8227be3d9a044ed888b2f54abd9463d', 'request_id': 'req-18e53ce6-3479-451d-b5d5-582e97925dcb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:06.777692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-18e53ce6-3479-451d-b5d5-582e97925dcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8227be3d9a044ed888b2f54abd9463d', 'request_id': 'req-18e53ce6-3479-451d-b5d5-582e97925dcb'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:06.789730 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-18e53ce6-3479-451d-b5d5-582e97925dcb admin admin] The token cache is being invalidate because project b8227be3d9a044ed888b2f54abd9463d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:47:06.790033 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-18e53ce6-3479-451d-b5d5-582e97925dcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:06.790699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-18e53ce6-3479-451d-b5d5-582e97925dcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:06.792535 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1400/2781] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:47:06 2022] DELETE /identity/v3/projects/b8227be3d9a044ed888b2f54abd9463d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:47:06.797667 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d476306-6c42-4eed-b063-51258b091474 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:06.797963 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d476306-6c42-4eed-b063-51258b091474 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:06.798251 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d476306-6c42-4eed-b063-51258b091474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:06.815877 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4d476306-6c42-4eed-b063-51258b091474 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:47:06.817139 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4d476306-6c42-4eed-b063-51258b091474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:06.875825 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1382/2782] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:47:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:47:06.883611 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c4b70-9481-4942-a5af-783a5cfeb80d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:06.883907 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c4b70-9481-4942-a5af-783a5cfeb80d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:06.884194 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c4b70-9481-4942-a5af-783a5cfeb80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:06.902227 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-e77c4b70-9481-4942-a5af-783a5cfeb80d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:47:06.903455 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-e77c4b70-9481-4942-a5af-783a5cfeb80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:06.911336 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1401/2783] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:47:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:47:06.915512 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d476306-6c42-4eed-b063-51258b091474 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:06.926190 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d476306-6c42-4eed-b063-51258b091474 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:06.930770 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4fb32b-395e-4862-af94-12af87fb49d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:06.931413 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fb32b-395e-4862-af94-12af87fb49d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:06.931698 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fb32b-395e-4862-af94-12af87fb49d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:06.931979 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fb32b-395e-4862-af94-12af87fb49d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:06.932705 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4fb32b-395e-4862-af94-12af87fb49d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:47:06.933682 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4fb32b-395e-4862-af94-12af87fb49d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:06.935352 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4fb32b-395e-4862-af94-12af87fb49d5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:06.936856 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:47:06.948745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1383/2784] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:47:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:47:06.952883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c4b70-9481-4942-a5af-783a5cfeb80d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:06.962007 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c4b70-9481-4942-a5af-783a5cfeb80d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:06.966880 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2e4fa-099a-44dd-8092-017909381031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:06.967515 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2e4fa-099a-44dd-8092-017909381031 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:06.967799 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2e4fa-099a-44dd-8092-017909381031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:06.968077 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2e4fa-099a-44dd-8092-017909381031 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:06.968760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e2e4fa-099a-44dd-8092-017909381031 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:06.970678 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e2e4fa-099a-44dd-8092-017909381031 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:07.005587 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1402/2785] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:47:06 2022] POST /identity/v3/projects => generated 403 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:47:07.009640 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4fb32b-395e-4862-af94-12af87fb49d5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:07.019648 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4fb32b-395e-4862-af94-12af87fb49d5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:07.075146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6bf011-a83e-4d25-a83b-b23116bfe472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:07.075719 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bf011-a83e-4d25-a83b-b23116bfe472 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:07.076004 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bf011-a83e-4d25-a83b-b23116bfe472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:07.076281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bf011-a83e-4d25-a83b-b23116bfe472 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:07.076968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6bf011-a83e-4d25-a83b-b23116bfe472 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:07.078956 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6bf011-a83e-4d25-a83b-b23116bfe472 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:07.099271 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1384/2786] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:47:07 2022] POST /identity/v3/users => generated 302 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:47:07.103493 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2e4fa-099a-44dd-8092-017909381031 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:07.114134 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2e4fa-099a-44dd-8092-017909381031 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:07.118719 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3b401-aee8-4de0-a786-bb87b1e86b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:07.119373 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3b401-aee8-4de0-a786-bb87b1e86b04 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:07.119663 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3b401-aee8-4de0-a786-bb87b1e86b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:07.119949 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3b401-aee8-4de0-a786-bb87b1e86b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:07.120739 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb3b401-aee8-4de0-a786-bb87b1e86b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:07.122386 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb3b401-aee8-4de0-a786-bb87b1e86b04 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:07.132120 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1403/2787] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:47:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:47:07.135910 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6bf011-a83e-4d25-a83b-b23116bfe472 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:07.146513 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6bf011-a83e-4d25-a83b-b23116bfe472 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:07.151126 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1cfef-c166-43e7-a329-129ddddac27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:07.152370 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1cfef-c166-43e7-a329-129ddddac27d admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:07.152651 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1cfef-c166-43e7-a329-129ddddac27d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:07.152924 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1cfef-c166-43e7-a329-129ddddac27d admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:07.153670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1cfef-c166-43e7-a329-129ddddac27d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:07.155255 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1cfef-c166-43e7-a329-129ddddac27d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:07.164886 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1385/2788] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:47:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:47:07.169256 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3b401-aee8-4de0-a786-bb87b1e86b04 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:07.179141 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3b401-aee8-4de0-a786-bb87b1e86b04 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:07.183821 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dbc8ca-fc84-44fb-9432-bb77d3e781fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:07.184693 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dbc8ca-fc84-44fb-9432-bb77d3e781fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:07.184987 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dbc8ca-fc84-44fb-9432-bb77d3e781fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:07.185302 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dbc8ca-fc84-44fb-9432-bb77d3e781fd admin admin] PATH_INFO: `/v3/projects/11c6667a26204fcd885e3a48e24afb34/users/57115d4f8d994a4087d64a364fdfa8cd/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:07.200645 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dbc8ca-fc84-44fb-9432-bb77d3e781fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=11c6667a26204fcd885e3a48e24afb34, user_id=57115d4f8d994a4087d64a364fdfa8cd, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:07.203332 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dbc8ca-fc84-44fb-9432-bb77d3e781fd admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:07.215793 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1404/2789] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:47:07 2022] PUT /identity/v3/projects/11c6667a26204fcd885e3a48e24afb34/users/57115d4f8d994a4087d64a364fdfa8cd/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:47:07.220777 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a04fe-0b9b-4154-a49a-43a4940cfc57 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:07.221077 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a04fe-0b9b-4154-a49a-43a4940cfc57 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:07.221388 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a04fe-0b9b-4154-a49a-43a4940cfc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:07.239362 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7f1a04fe-0b9b-4154-a49a-43a4940cfc57 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:47:07.240557 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7f1a04fe-0b9b-4154-a49a-43a4940cfc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:07.303491 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1386/2790] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:47:07.312720 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dbc8ca-fc84-44fb-9432-bb77d3e781fd admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:07.314639 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b8dbc8ca-fc84-44fb-9432-bb77d3e781fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:07.321263 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:47:07.377736 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dbc8ca-fc84-44fb-9432-bb77d3e781fd admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:07.381476 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe110e3-19a4-4322-9ae5-763f3c9a9123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:07.382156 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe110e3-19a4-4322-9ae5-763f3c9a9123 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:07.382448 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe110e3-19a4-4322-9ae5-763f3c9a9123 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:07.382734 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe110e3-19a4-4322-9ae5-763f3c9a9123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:07.392319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe110e3-19a4-4322-9ae5-763f3c9a9123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:07.393795 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe110e3-19a4-4322-9ae5-763f3c9a9123 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:07.403393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1405/2791] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:47:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:47:08.004001 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5508e0-6913-4744-a682-c6730cf063fa None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:08.004570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5508e0-6913-4744-a682-c6730cf063fa None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:08.004570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5508e0-6913-4744-a682-c6730cf063fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:08.033502 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-7c5508e0-6913-4744-a682-c6730cf063fa None None] MFA Rules not processed for user `57115d4f8d994a4087d64a364fdfa8cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:47:08.034825 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-7c5508e0-6913-4744-a682-c6730cf063fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:08.106183 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1387/2792] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:47:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:47:08.110510 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe110e3-19a4-4322-9ae5-763f3c9a9123 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:08.122579 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe110e3-19a4-4322-9ae5-763f3c9a9123 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:08.127877 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2817d-95c7-4100-a60e-d4ee2603f28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:08.129095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2817d-95c7-4100-a60e-d4ee2603f28d admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:08.129798 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2817d-95c7-4100-a60e-d4ee2603f28d admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:08.130319 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2817d-95c7-4100-a60e-d4ee2603f28d admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:08.131337 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2817d-95c7-4100-a60e-d4ee2603f28d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:08.133534 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2817d-95c7-4100-a60e-d4ee2603f28d admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:08.173281 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1406/2793] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:47:08 2022] POST /identity/v3/projects => generated 403 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:47:08.177079 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5508e0-6913-4744-a682-c6730cf063fa None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:08.182468 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:47:08.192049 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5508e0-6913-4744-a682-c6730cf063fa None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:08.264760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d77311-0654-438c-a190-347a4e92d753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:08.265437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d77311-0654-438c-a190-347a4e92d753 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:08.265763 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d77311-0654-438c-a190-347a4e92d753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:08.266048 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d77311-0654-438c-a190-347a4e92d753 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:08.266743 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d77311-0654-438c-a190-347a4e92d753 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:08.268743 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d77311-0654-438c-a190-347a4e92d753 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:08.294225 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1388/2794] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:47:08 2022] POST /identity/v3/users => generated 308 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:47:08.298165 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2817d-95c7-4100-a60e-d4ee2603f28d admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:08.308679 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2817d-95c7-4100-a60e-d4ee2603f28d admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:08.313361 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410e7a2-8f93-4b39-a0de-8733abf9440a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:08.314097 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410e7a2-8f93-4b39-a0de-8733abf9440a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:08.314388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410e7a2-8f93-4b39-a0de-8733abf9440a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:08.314676 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410e7a2-8f93-4b39-a0de-8733abf9440a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:08.315409 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410e7a2-8f93-4b39-a0de-8733abf9440a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:08.317026 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410e7a2-8f93-4b39-a0de-8733abf9440a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:08.332983 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1407/2795] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:47:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:47:08.334647 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d77311-0654-438c-a190-347a4e92d753 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:08.345780 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d77311-0654-438c-a190-347a4e92d753 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:08.350494 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906c4e3-8b94-4f7e-87a9-c0e1a107c8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:08.351396 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906c4e3-8b94-4f7e-87a9-c0e1a107c8b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:08.351679 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906c4e3-8b94-4f7e-87a9-c0e1a107c8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:08.351958 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906c4e3-8b94-4f7e-87a9-c0e1a107c8b4 admin admin] PATH_INFO: `/v3/projects/0710bafe3b7a43d591f661cffa2f28cd/users/eaafc37bd1dd4905b94f4e94365b456e/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:08.367682 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906c4e3-8b94-4f7e-87a9-c0e1a107c8b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0710bafe3b7a43d591f661cffa2f28cd, user_id=eaafc37bd1dd4905b94f4e94365b456e, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:08.370476 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906c4e3-8b94-4f7e-87a9-c0e1a107c8b4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:08.387868 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1389/2796] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:47:08 2022] PUT /identity/v3/projects/0710bafe3b7a43d591f661cffa2f28cd/users/eaafc37bd1dd4905b94f4e94365b456e/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:47:08.391872 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410e7a2-8f93-4b39-a0de-8733abf9440a admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:08.402561 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410e7a2-8f93-4b39-a0de-8733abf9440a admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:08.458865 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54222d3d-806b-4718-9948-bdce76647278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:08.459542 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54222d3d-806b-4718-9948-bdce76647278 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:08.459830 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54222d3d-806b-4718-9948-bdce76647278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:08.460118 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54222d3d-806b-4718-9948-bdce76647278 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:08.460847 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54222d3d-806b-4718-9948-bdce76647278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:08.462494 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54222d3d-806b-4718-9948-bdce76647278 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:08.473039 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1408/2797] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:47:08 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:47:08.477080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906c4e3-8b94-4f7e-87a9-c0e1a107c8b4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:08.488884 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906c4e3-8b94-4f7e-87a9-c0e1a107c8b4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:08.494414 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd27871-f583-413d-85a7-2880f8ac121b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:08.495391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd27871-f583-413d-85a7-2880f8ac121b admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:08.496404 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd27871-f583-413d-85a7-2880f8ac121b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:08.496947 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd27871-f583-413d-85a7-2880f8ac121b admin admin] PATH_INFO: `/v3/projects/0710bafe3b7a43d591f661cffa2f28cd/users/eaafc37bd1dd4905b94f4e94365b456e/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:08.498638 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd27871-f583-413d-85a7-2880f8ac121b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0710bafe3b7a43d591f661cffa2f28cd, user_id=eaafc37bd1dd4905b94f4e94365b456e, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:08.501330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd27871-f583-413d-85a7-2880f8ac121b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:08.513338 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1390/2798] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:47:08 2022] PUT /identity/v3/projects/0710bafe3b7a43d591f661cffa2f28cd/users/eaafc37bd1dd4905b94f4e94365b456e/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:47:09.022287 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54222d3d-806b-4718-9948-bdce76647278 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:09.024621 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-54222d3d-806b-4718-9948-bdce76647278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:09.025359 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd27871-f583-413d-85a7-2880f8ac121b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:09.027445 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-fdd27871-f583-413d-85a7-2880f8ac121b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:09.118111 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd27871-f583-413d-85a7-2880f8ac121b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:09.129602 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a91e1a-22ef-45a9-aa3b-df5d41659872 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:09.130901 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a91e1a-22ef-45a9-aa3b-df5d41659872 service nova] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:09.131197 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a91e1a-22ef-45a9-aa3b-df5d41659872 service nova] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:09.131611 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a91e1a-22ef-45a9-aa3b-df5d41659872 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:09.135695 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-69a91e1a-22ef-45a9-aa3b-df5d41659872 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:09.150960 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a91e1a-22ef-45a9-aa3b-df5d41659872 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:09.152762 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a91e1a-22ef-45a9-aa3b-df5d41659872 service nova] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:09.166573 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1391/2799] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:47:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:47:09.179941 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54222d3d-806b-4718-9948-bdce76647278 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:09.190756 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df928c27-a51b-4801-8542-2fdfe64a9a03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8751a4e92174d18b7b138392d2579a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:09.191591 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df928c27-a51b-4801-8542-2fdfe64a9a03 service nova] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:09.192221 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df928c27-a51b-4801-8542-2fdfe64a9a03 service nova] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:09.192608 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df928c27-a51b-4801-8542-2fdfe64a9a03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:09.195637 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-df928c27-a51b-4801-8542-2fdfe64a9a03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:09.206197 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:47:09.252382 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727660a-8fda-4e96-98bf-2564ca2323d5 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:09.252698 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727660a-8fda-4e96-98bf-2564ca2323d5 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:09.252983 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727660a-8fda-4e96-98bf-2564ca2323d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:09.300223 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3727660a-8fda-4e96-98bf-2564ca2323d5 None None] MFA Rules not processed for user `eaafc37bd1dd4905b94f4e94365b456e`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:47:09.305727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3727660a-8fda-4e96-98bf-2564ca2323d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:09.330486 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df928c27-a51b-4801-8542-2fdfe64a9a03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:09.332225 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df928c27-a51b-4801-8542-2fdfe64a9a03 service nova] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:09.343056 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1409/2800] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:47:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:47:09.350428 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:47:09.445768 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1392/2801] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:47:09 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:47:10.930042 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df928c27-a51b-4801-8542-2fdfe64a9a03 service nova] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:10.931490 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-df928c27-a51b-4801-8542-2fdfe64a9a03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:11.005431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df928c27-a51b-4801-8542-2fdfe64a9a03 service nova] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:11.012260 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e25a2-bbc5-4177-b27f-5395274feb2e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49325c1733174e0c98b3475a10c9b5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:11.012940 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e25a2-bbc5-4177-b27f-5395274feb2e service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:11.013229 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e25a2-bbc5-4177-b27f-5395274feb2e service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:11.013510 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e25a2-bbc5-4177-b27f-5395274feb2e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:11.015330 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-916e25a2-bbc5-4177-b27f-5395274feb2e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:11.028602 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e25a2-bbc5-4177-b27f-5395274feb2e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:11.030183 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e25a2-bbc5-4177-b27f-5395274feb2e service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:11.043379 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1410/2802] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:47:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:47:11.044880 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727660a-8fda-4e96-98bf-2564ca2323d5 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:11.125217 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727660a-8fda-4e96-98bf-2564ca2323d5 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:11.129800 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4b50c-81dc-432d-916f-4320f3997cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:11.130664 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4b50c-81dc-432d-916f-4320f3997cf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:11.130965 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4b50c-81dc-432d-916f-4320f3997cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:11.131235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4b50c-81dc-432d-916f-4320f3997cf6 admin admin] PATH_INFO: `/v3/users/57115d4f8d994a4087d64a364fdfa8cd` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:11.132435 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4b50c-81dc-432d-916f-4320f3997cf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57115d4f8d994a4087d64a364fdfa8cd)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:11.134393 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4b50c-81dc-432d-916f-4320f3997cf6 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:11.182481 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4fa4b50c-81dc-432d-916f-4320f3997cf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57115d4f8d994a4087d64a364fdfa8cd'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:11.183178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4fa4b50c-81dc-432d-916f-4320f3997cf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57115d4f8d994a4087d64a364fdfa8cd'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:11.195094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4fa4b50c-81dc-432d-916f-4320f3997cf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57115d4f8d994a4087d64a364fdfa8cd'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:11.201793 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4fa4b50c-81dc-432d-916f-4320f3997cf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57115d4f8d994a4087d64a364fdfa8cd'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:11.202419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-4fa4b50c-81dc-432d-916f-4320f3997cf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57115d4f8d994a4087d64a364fdfa8cd'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:11.214832 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1393/2803] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:47:11 2022] DELETE /identity/v3/users/57115d4f8d994a4087d64a364fdfa8cd => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:47:11.399787 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e25a2-bbc5-4177-b27f-5395274feb2e service placement] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:11.412515 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e25a2-bbc5-4177-b27f-5395274feb2e service placement] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:11.472684 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a554f9-b3c4-49fe-9f1b-eac2d5bac565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:11.473549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a554f9-b3c4-49fe-9f1b-eac2d5bac565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:11.473873 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a554f9-b3c4-49fe-9f1b-eac2d5bac565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:11.474156 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a554f9-b3c4-49fe-9f1b-eac2d5bac565 admin admin] PATH_INFO: `/v3/projects/11c6667a26204fcd885e3a48e24afb34` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:11.475454 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a554f9-b3c4-49fe-9f1b-eac2d5bac565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11c6667a26204fcd885e3a48e24afb34)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:11.477433 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a554f9-b3c4-49fe-9f1b-eac2d5bac565 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:11.519857 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:47:11.527897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f8a554f9-b3c4-49fe-9f1b-eac2d5bac565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11c6667a26204fcd885e3a48e24afb34', 'request_id': 'req-f8a554f9-b3c4-49fe-9f1b-eac2d5bac565'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:11.534395 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f8a554f9-b3c4-49fe-9f1b-eac2d5bac565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11c6667a26204fcd885e3a48e24afb34', 'request_id': 'req-f8a554f9-b3c4-49fe-9f1b-eac2d5bac565'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:11.539921 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f8a554f9-b3c4-49fe-9f1b-eac2d5bac565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11c6667a26204fcd885e3a48e24afb34', 'request_id': 'req-f8a554f9-b3c4-49fe-9f1b-eac2d5bac565'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:11.554586 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f8a554f9-b3c4-49fe-9f1b-eac2d5bac565 admin admin] The token cache is being invalidate because project 11c6667a26204fcd885e3a48e24afb34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:47:11.554884 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f8a554f9-b3c4-49fe-9f1b-eac2d5bac565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:11.555523 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-f8a554f9-b3c4-49fe-9f1b-eac2d5bac565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:11.557076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1411/2804] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:47:11 2022] DELETE /identity/v3/projects/11c6667a26204fcd885e3a48e24afb34 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:47:11.561373 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4b50c-81dc-432d-916f-4320f3997cf6 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:11.562875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4fa4b50c-81dc-432d-916f-4320f3997cf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:11.576053 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4b50c-81dc-432d-916f-4320f3997cf6 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:11.621699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:47:11.643968 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91ce690-1a86-4baf-9154-843ef3691fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:11.644844 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ce690-1a86-4baf-9154-843ef3691fc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:11.645129 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ce690-1a86-4baf-9154-843ef3691fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:11.645410 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ce690-1a86-4baf-9154-843ef3691fc1 admin admin] PATH_INFO: `/v3/users/eaafc37bd1dd4905b94f4e94365b456e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:11.646746 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91ce690-1a86-4baf-9154-843ef3691fc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaafc37bd1dd4905b94f4e94365b456e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:11.648713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91ce690-1a86-4baf-9154-843ef3691fc1 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:11.698131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b91ce690-1a86-4baf-9154-843ef3691fc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaafc37bd1dd4905b94f4e94365b456e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:11.699173 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b91ce690-1a86-4baf-9154-843ef3691fc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaafc37bd1dd4905b94f4e94365b456e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:11.711713 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b91ce690-1a86-4baf-9154-843ef3691fc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaafc37bd1dd4905b94f4e94365b456e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:11.720767 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b91ce690-1a86-4baf-9154-843ef3691fc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaafc37bd1dd4905b94f4e94365b456e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:11.721410 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-b91ce690-1a86-4baf-9154-843ef3691fc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaafc37bd1dd4905b94f4e94365b456e'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:11.736001 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1394/2805] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:47:11 2022] DELETE /identity/v3/users/eaafc37bd1dd4905b94f4e94365b456e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:47:11.921462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a554f9-b3c4-49fe-9f1b-eac2d5bac565 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:11.925391 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-f8a554f9-b3c4-49fe-9f1b-eac2d5bac565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:11.941307 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a554f9-b3c4-49fe-9f1b-eac2d5bac565 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:12.003002 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ed722-f884-45b5-a515-78ebe0cbeba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:12.003866 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ed722-f884-45b5-a515-78ebe0cbeba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:12.004157 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ed722-f884-45b5-a515-78ebe0cbeba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:12.004440 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ed722-f884-45b5-a515-78ebe0cbeba1 admin admin] PATH_INFO: `/v3/projects/0710bafe3b7a43d591f661cffa2f28cd` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:12.005649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ed722-f884-45b5-a515-78ebe0cbeba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0710bafe3b7a43d591f661cffa2f28cd)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:12.007651 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ed722-f884-45b5-a515-78ebe0cbeba1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:12.053525 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ca9ed722-f884-45b5-a515-78ebe0cbeba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0710bafe3b7a43d591f661cffa2f28cd', 'request_id': 'req-ca9ed722-f884-45b5-a515-78ebe0cbeba1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:12.060752 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ca9ed722-f884-45b5-a515-78ebe0cbeba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0710bafe3b7a43d591f661cffa2f28cd', 'request_id': 'req-ca9ed722-f884-45b5-a515-78ebe0cbeba1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:12.066344 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ca9ed722-f884-45b5-a515-78ebe0cbeba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0710bafe3b7a43d591f661cffa2f28cd', 'request_id': 'req-ca9ed722-f884-45b5-a515-78ebe0cbeba1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:12.080086 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ca9ed722-f884-45b5-a515-78ebe0cbeba1 admin admin] The token cache is being invalidate because project 0710bafe3b7a43d591f661cffa2f28cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:47:12.080447 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ca9ed722-f884-45b5-a515-78ebe0cbeba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:12.081096 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ca9ed722-f884-45b5-a515-78ebe0cbeba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:12.082742 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1412/2806] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:47:11 2022] DELETE /identity/v3/projects/0710bafe3b7a43d591f661cffa2f28cd => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:47:12.087571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c118ad-11a2-4a16-88bc-5bc8b760e65a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:12.087988 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c118ad-11a2-4a16-88bc-5bc8b760e65a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:12.088094 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c118ad-11a2-4a16-88bc-5bc8b760e65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:12.106645 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-82c118ad-11a2-4a16-88bc-5bc8b760e65a None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:47:12.107831 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-82c118ad-11a2-4a16-88bc-5bc8b760e65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:12.192669 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1395/2807] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:47:12.200791 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784686f-a09e-4b74-949e-68d171356c5d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:12.201136 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784686f-a09e-4b74-949e-68d171356c5d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:12.201454 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784686f-a09e-4b74-949e-68d171356c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:12.226856 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-8784686f-a09e-4b74-949e-68d171356c5d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:47:12.228259 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-8784686f-a09e-4b74-949e-68d171356c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:12.237424 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1413/2808] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:47:12.242425 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c118ad-11a2-4a16-88bc-5bc8b760e65a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:12.253490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c118ad-11a2-4a16-88bc-5bc8b760e65a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:12.258680 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18e7c5-e169-47f8-b8cb-b635cd786b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:12.259631 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18e7c5-e169-47f8-b8cb-b635cd786b4c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:12.259936 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18e7c5-e169-47f8-b8cb-b635cd786b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:12.260185 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18e7c5-e169-47f8-b8cb-b635cd786b4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:12.260946 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18e7c5-e169-47f8-b8cb-b635cd786b4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:47:12.261252 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18e7c5-e169-47f8-b8cb-b635cd786b4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:12.263338 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18e7c5-e169-47f8-b8cb-b635cd786b4c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:12.280201 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1396/2809] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:47:12.281172 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784686f-a09e-4b74-949e-68d171356c5d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:12.291765 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784686f-a09e-4b74-949e-68d171356c5d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:12.297353 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f96fa2-0bea-4bcd-86f5-00a5923272a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:12.298074 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f96fa2-0bea-4bcd-86f5-00a5923272a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:12.298365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f96fa2-0bea-4bcd-86f5-00a5923272a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:12.298700 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f96fa2-0bea-4bcd-86f5-00a5923272a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:12.299387 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f96fa2-0bea-4bcd-86f5-00a5923272a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:12.301844 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f96fa2-0bea-4bcd-86f5-00a5923272a8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:12.337260 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1414/2810] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:47:12 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:47:12.342571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18e7c5-e169-47f8-b8cb-b635cd786b4c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:12.353839 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18e7c5-e169-47f8-b8cb-b635cd786b4c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:12.401330 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:47:12.413452 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3a73a-0ddf-4d1b-a02b-693bc5f07043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:12.414095 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3a73a-0ddf-4d1b-a02b-693bc5f07043 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:12.414384 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3a73a-0ddf-4d1b-a02b-693bc5f07043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:12.414665 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3a73a-0ddf-4d1b-a02b-693bc5f07043 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:12.415358 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a3a73a-0ddf-4d1b-a02b-693bc5f07043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:12.417982 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a3a73a-0ddf-4d1b-a02b-693bc5f07043 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:12.437181 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1397/2811] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:47:12 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:47:12.441309 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f96fa2-0bea-4bcd-86f5-00a5923272a8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:12.452142 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f96fa2-0bea-4bcd-86f5-00a5923272a8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:12.456810 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a411f6-0e36-4f00-803f-a00fc34372a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:12.457463 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a411f6-0e36-4f00-803f-a00fc34372a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:12.457793 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a411f6-0e36-4f00-803f-a00fc34372a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:12.458110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a411f6-0e36-4f00-803f-a00fc34372a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:12.458829 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a411f6-0e36-4f00-803f-a00fc34372a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:12.460431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a411f6-0e36-4f00-803f-a00fc34372a9 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:12.471622 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1415/2812] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:47:12.475579 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3a73a-0ddf-4d1b-a02b-693bc5f07043 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:12.486249 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3a73a-0ddf-4d1b-a02b-693bc5f07043 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:12.490865 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b2a96-e423-4840-999d-fa9c9709ce65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:12.491540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b2a96-e423-4840-999d-fa9c9709ce65 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:12.491830 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b2a96-e423-4840-999d-fa9c9709ce65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:12.492110 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b2a96-e423-4840-999d-fa9c9709ce65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:12.492839 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b2a96-e423-4840-999d-fa9c9709ce65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:12.494470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b2a96-e423-4840-999d-fa9c9709ce65 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:12.505308 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1398/2813] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:47:12.509337 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a411f6-0e36-4f00-803f-a00fc34372a9 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:12.520019 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a411f6-0e36-4f00-803f-a00fc34372a9 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:12.524652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9c4e4-b423-4572-a2a7-abece2c0c91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:12.525527 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9c4e4-b423-4572-a2a7-abece2c0c91e admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:12.525842 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9c4e4-b423-4572-a2a7-abece2c0c91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:12.526124 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9c4e4-b423-4572-a2a7-abece2c0c91e admin admin] PATH_INFO: `/v3/projects/59e9084b8f704da2be0721b548dfb781/users/b82f5ce61ad742f888cd89a07e7cdb91/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:12.540929 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9c4e4-b423-4572-a2a7-abece2c0c91e admin admin] RBAC: Authorizing `identity:create_grant(project_id=59e9084b8f704da2be0721b548dfb781, user_id=b82f5ce61ad742f888cd89a07e7cdb91, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:12.544589 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9c4e4-b423-4572-a2a7-abece2c0c91e admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:12.546885 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:47:12.560006 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1416/2814] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:47:12 2022] PUT /identity/v3/projects/59e9084b8f704da2be0721b548dfb781/users/b82f5ce61ad742f888cd89a07e7cdb91/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:47:12.571398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584247e-0ce9-4535-81d8-6529d287fcbb None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:12.571691 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584247e-0ce9-4535-81d8-6529d287fcbb None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:12.572004 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584247e-0ce9-4535-81d8-6529d287fcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:12.602264 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a584247e-0ce9-4535-81d8-6529d287fcbb None None] MFA Rules not processed for user `b82f5ce61ad742f888cd89a07e7cdb91`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:47:12.603416 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a584247e-0ce9-4535-81d8-6529d287fcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:12.677912 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1399/2815] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:47:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:47:12.683693 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9c4e4-b423-4572-a2a7-abece2c0c91e admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:12.750042 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9c4e4-b423-4572-a2a7-abece2c0c91e admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:12.754753 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc274a-143a-4e81-9400-c239df2f04df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:12.755413 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc274a-143a-4e81-9400-c239df2f04df admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:12.755696 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc274a-143a-4e81-9400-c239df2f04df admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:12.755974 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc274a-143a-4e81-9400-c239df2f04df admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:12.756652 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc274a-143a-4e81-9400-c239df2f04df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:12.758599 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc274a-143a-4e81-9400-c239df2f04df admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:12.786364 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1417/2816] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:47:12 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:47:12.789860 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a584247e-0ce9-4535-81d8-6529d287fcbb None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:12.803569 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a584247e-0ce9-4535-81d8-6529d287fcbb None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:12.860980 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50885d8-0085-4831-ad90-9d7e0c9233c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:12.861571 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50885d8-0085-4831-ad90-9d7e0c9233c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:12.861885 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50885d8-0085-4831-ad90-9d7e0c9233c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:12.862166 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50885d8-0085-4831-ad90-9d7e0c9233c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:12.862853 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50885d8-0085-4831-ad90-9d7e0c9233c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:12.864783 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50885d8-0085-4831-ad90-9d7e0c9233c5 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:12.884976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1400/2817] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:47:12 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:47:12.889693 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc274a-143a-4e81-9400-c239df2f04df admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:12.899995 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc274a-143a-4e81-9400-c239df2f04df admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:12.905142 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d481bcc-b712-4a9c-a637-6779f6ffc160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:12.905824 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d481bcc-b712-4a9c-a637-6779f6ffc160 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:12.906110 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d481bcc-b712-4a9c-a637-6779f6ffc160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:12.906389 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d481bcc-b712-4a9c-a637-6779f6ffc160 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:12.907104 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d481bcc-b712-4a9c-a637-6779f6ffc160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:12.908680 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d481bcc-b712-4a9c-a637-6779f6ffc160 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:12.919385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1418/2818] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:47:12.923417 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50885d8-0085-4831-ad90-9d7e0c9233c5 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:12.933377 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50885d8-0085-4831-ad90-9d7e0c9233c5 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:12.937959 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6393462-7bbf-405e-b1ac-eeab205cca00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:12.938860 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6393462-7bbf-405e-b1ac-eeab205cca00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:12.939316 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6393462-7bbf-405e-b1ac-eeab205cca00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:12.939611 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6393462-7bbf-405e-b1ac-eeab205cca00 admin admin] PATH_INFO: `/v3/projects/dc533bf24c584bac872ada7bdc8e9076/users/2bc5f4636f3542caba1dbf963d947a89/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:12.954654 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6393462-7bbf-405e-b1ac-eeab205cca00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc533bf24c584bac872ada7bdc8e9076, user_id=2bc5f4636f3542caba1dbf963d947a89, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:12.957350 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6393462-7bbf-405e-b1ac-eeab205cca00 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:12.970515 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1401/2819] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:47:12 2022] PUT /identity/v3/projects/dc533bf24c584bac872ada7bdc8e9076/users/2bc5f4636f3542caba1dbf963d947a89/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:47:12.974424 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d481bcc-b712-4a9c-a637-6779f6ffc160 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:12.984660 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d481bcc-b712-4a9c-a637-6779f6ffc160 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:13.040266 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a1f4c-65aa-42ac-87c8-ec0b6fb49291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:13.040929 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a1f4c-65aa-42ac-87c8-ec0b6fb49291 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:13.041214 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a1f4c-65aa-42ac-87c8-ec0b6fb49291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:13.041493 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a1f4c-65aa-42ac-87c8-ec0b6fb49291 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:13.042670 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a1f4c-65aa-42ac-87c8-ec0b6fb49291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:13.044423 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a1f4c-65aa-42ac-87c8-ec0b6fb49291 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:13.054314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1419/2820] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:47:13.058408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6393462-7bbf-405e-b1ac-eeab205cca00 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:13.068879 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6393462-7bbf-405e-b1ac-eeab205cca00 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:13.074061 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d107aac-3c7f-48f8-9361-a0f0047e829d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:13.075015 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d107aac-3c7f-48f8-9361-a0f0047e829d admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:13.075299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d107aac-3c7f-48f8-9361-a0f0047e829d admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:13.075581 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d107aac-3c7f-48f8-9361-a0f0047e829d admin admin] PATH_INFO: `/v3/projects/dc533bf24c584bac872ada7bdc8e9076/users/2bc5f4636f3542caba1dbf963d947a89/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:13.077563 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d107aac-3c7f-48f8-9361-a0f0047e829d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc533bf24c584bac872ada7bdc8e9076, user_id=2bc5f4636f3542caba1dbf963d947a89, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:13.080257 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d107aac-3c7f-48f8-9361-a0f0047e829d admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:13.091604 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1402/2821] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:47:13 2022] PUT /identity/v3/projects/dc533bf24c584bac872ada7bdc8e9076/users/2bc5f4636f3542caba1dbf963d947a89/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:47:13.101686 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10c268-714b-4fe3-a687-af900ed930fe None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:13.102020 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10c268-714b-4fe3-a687-af900ed930fe None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:13.102342 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10c268-714b-4fe3-a687-af900ed930fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:13.129570 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-ee10c268-714b-4fe3-a687-af900ed930fe None None] MFA Rules not processed for user `2bc5f4636f3542caba1dbf963d947a89`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:47:13.130744 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ee10c268-714b-4fe3-a687-af900ed930fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:13.181125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:47:13.202095 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1420/2822] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:47:13 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:47:13.213700 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d107aac-3c7f-48f8-9361-a0f0047e829d admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:13.216006 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9d107aac-3c7f-48f8-9361-a0f0047e829d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:13.236965 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:47:13.287952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d107aac-3c7f-48f8-9361-a0f0047e829d admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:13.292346 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e1dcb7-950a-4758-bb10-fa20d9a788f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:13.293153 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1dcb7-950a-4758-bb10-fa20d9a788f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:13.293434 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1dcb7-950a-4758-bb10-fa20d9a788f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:13.293745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1dcb7-950a-4758-bb10-fa20d9a788f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:13.303704 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e1dcb7-950a-4758-bb10-fa20d9a788f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:13.305155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e1dcb7-950a-4758-bb10-fa20d9a788f8 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:13.315725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1403/2823] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:47:13 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:47:13.393530 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10c268-714b-4fe3-a687-af900ed930fe None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:13.407441 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10c268-714b-4fe3-a687-af900ed930fe None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:13.412184 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315123f6-c630-4496-844d-c9561df29c6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:13.413309 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315123f6-c630-4496-844d-c9561df29c6e service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:13.413876 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315123f6-c630-4496-844d-c9561df29c6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:13.414291 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315123f6-c630-4496-844d-c9561df29c6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:13.445059 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e1dcb7-950a-4758-bb10-fa20d9a788f8 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:13.446665 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-89e1dcb7-950a-4758-bb10-fa20d9a788f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:13.485776 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315123f6-c630-4496-844d-c9561df29c6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:13.487241 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315123f6-c630-4496-844d-c9561df29c6e service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:13.496419 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1421/2824] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:47:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:47:13.508537 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e1dcb7-950a-4758-bb10-fa20d9a788f8 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:13.512845 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b4c23-d3ed-457b-98e8-2a089ac5b3e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:13.513500 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b4c23-d3ed-457b-98e8-2a089ac5b3e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:13.513794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b4c23-d3ed-457b-98e8-2a089ac5b3e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:13.514072 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b4c23-d3ed-457b-98e8-2a089ac5b3e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:13.515841 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-180b4c23-d3ed-457b-98e8-2a089ac5b3e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:13.590858 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b4c23-d3ed-457b-98e8-2a089ac5b3e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:13.592356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b4c23-d3ed-457b-98e8-2a089ac5b3e2 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:13.608288 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1404/2825] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:47:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:47:19.731429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315123f6-c630-4496-844d-c9561df29c6e service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:19.733091 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-315123f6-c630-4496-844d-c9561df29c6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:19.751268 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315123f6-c630-4496-844d-c9561df29c6e service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:19.755942 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae79119-b546-4d4a-93ae-247dbd2d7fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:19.756878 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae79119-b546-4d4a-93ae-247dbd2d7fe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:19.757162 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae79119-b546-4d4a-93ae-247dbd2d7fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:19.757438 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae79119-b546-4d4a-93ae-247dbd2d7fe1 admin admin] PATH_INFO: `/v3/users/f7f4e45a0882410088464b550a4b06c1` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:19.758743 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae79119-b546-4d4a-93ae-247dbd2d7fe1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7f4e45a0882410088464b550a4b06c1)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:19.760796 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae79119-b546-4d4a-93ae-247dbd2d7fe1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:19.815449 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9ae79119-b546-4d4a-93ae-247dbd2d7fe1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7f4e45a0882410088464b550a4b06c1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:19.827106 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9ae79119-b546-4d4a-93ae-247dbd2d7fe1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7f4e45a0882410088464b550a4b06c1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:19.827774 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9ae79119-b546-4d4a-93ae-247dbd2d7fe1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7f4e45a0882410088464b550a4b06c1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:19.834388 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9ae79119-b546-4d4a-93ae-247dbd2d7fe1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7f4e45a0882410088464b550a4b06c1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:19.835008 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-9ae79119-b546-4d4a-93ae-247dbd2d7fe1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7f4e45a0882410088464b550a4b06c1'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:19.848166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1422/2826] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:47:19 2022] DELETE /identity/v3/users/f7f4e45a0882410088464b550a4b06c1 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:47:20.020634 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b4c23-d3ed-457b-98e8-2a089ac5b3e2 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:20.031336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b4c23-d3ed-457b-98e8-2a089ac5b3e2 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:20.091404 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c888a-295a-44c3-a055-9194562ef4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:20.092259 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c888a-295a-44c3-a055-9194562ef4ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:20.092544 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c888a-295a-44c3-a055-9194562ef4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:20.092827 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c888a-295a-44c3-a055-9194562ef4ed admin admin] PATH_INFO: `/v3/projects/8bc355242c9d490d91fbcbd7d08f8b97` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:20.094044 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c888a-295a-44c3-a055-9194562ef4ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bc355242c9d490d91fbcbd7d08f8b97)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:20.095975 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c888a-295a-44c3-a055-9194562ef4ed admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:20.102081 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=100 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:47:20.141486 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a53c888a-295a-44c3-a055-9194562ef4ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bc355242c9d490d91fbcbd7d08f8b97', 'request_id': 'req-a53c888a-295a-44c3-a055-9194562ef4ed'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:20.147489 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a53c888a-295a-44c3-a055-9194562ef4ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bc355242c9d490d91fbcbd7d08f8b97', 'request_id': 'req-a53c888a-295a-44c3-a055-9194562ef4ed'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:20.153444 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a53c888a-295a-44c3-a055-9194562ef4ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bc355242c9d490d91fbcbd7d08f8b97', 'request_id': 'req-a53c888a-295a-44c3-a055-9194562ef4ed'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:20.165382 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a53c888a-295a-44c3-a055-9194562ef4ed admin admin] The token cache is being invalidate because project 8bc355242c9d490d91fbcbd7d08f8b97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:47:20.165696 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a53c888a-295a-44c3-a055-9194562ef4ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:20.166312 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-a53c888a-295a-44c3-a055-9194562ef4ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:20.168505 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1405/2827] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:47:20 2022] DELETE /identity/v3/projects/8bc355242c9d490d91fbcbd7d08f8b97 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:47:20.172537 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae79119-b546-4d4a-93ae-247dbd2d7fe1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:20.173963 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-9ae79119-b546-4d4a-93ae-247dbd2d7fe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:20.186940 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae79119-b546-4d4a-93ae-247dbd2d7fe1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:20.242716 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31e367-8982-48a2-ab1d-c0017e25f5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:20.243577 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31e367-8982-48a2-ab1d-c0017e25f5f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:20.243854 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31e367-8982-48a2-ab1d-c0017e25f5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:20.244125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31e367-8982-48a2-ab1d-c0017e25f5f5 admin admin] PATH_INFO: `/v3/users/992acfd6168845a7817a7aa15fdcb1e3` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:20.245861 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31e367-8982-48a2-ab1d-c0017e25f5f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=992acfd6168845a7817a7aa15fdcb1e3)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:20.247767 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31e367-8982-48a2-ab1d-c0017e25f5f5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:20.249179 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:47:20.296508 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ce31e367-8982-48a2-ab1d-c0017e25f5f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '992acfd6168845a7817a7aa15fdcb1e3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:20.308650 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ce31e367-8982-48a2-ab1d-c0017e25f5f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '992acfd6168845a7817a7aa15fdcb1e3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:20.309329 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ce31e367-8982-48a2-ab1d-c0017e25f5f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '992acfd6168845a7817a7aa15fdcb1e3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:20.316167 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ce31e367-8982-48a2-ab1d-c0017e25f5f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '992acfd6168845a7817a7aa15fdcb1e3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:20.316791 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ce31e367-8982-48a2-ab1d-c0017e25f5f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '992acfd6168845a7817a7aa15fdcb1e3'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:20.329365 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1423/2828] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:47:20 2022] DELETE /identity/v3/users/992acfd6168845a7817a7aa15fdcb1e3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:47:20.509212 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c888a-295a-44c3-a055-9194562ef4ed admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:20.511308 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a53c888a-295a-44c3-a055-9194562ef4ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:20.535889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c888a-295a-44c3-a055-9194562ef4ed admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:20.599230 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba19c363-c78e-4147-9efb-851735a91b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:20.600086 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba19c363-c78e-4147-9efb-851735a91b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:20.600375 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba19c363-c78e-4147-9efb-851735a91b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:20.600654 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba19c363-c78e-4147-9efb-851735a91b5b admin admin] PATH_INFO: `/v3/projects/5a6986baffaf4f3a842ea1fe2a974741` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:20.601875 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba19c363-c78e-4147-9efb-851735a91b5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a6986baffaf4f3a842ea1fe2a974741)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:20.603924 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba19c363-c78e-4147-9efb-851735a91b5b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:20.648364 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ba19c363-c78e-4147-9efb-851735a91b5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a6986baffaf4f3a842ea1fe2a974741', 'request_id': 'req-ba19c363-c78e-4147-9efb-851735a91b5b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:20.655211 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ba19c363-c78e-4147-9efb-851735a91b5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a6986baffaf4f3a842ea1fe2a974741', 'request_id': 'req-ba19c363-c78e-4147-9efb-851735a91b5b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:20.660682 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ba19c363-c78e-4147-9efb-851735a91b5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a6986baffaf4f3a842ea1fe2a974741', 'request_id': 'req-ba19c363-c78e-4147-9efb-851735a91b5b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:20.672839 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ba19c363-c78e-4147-9efb-851735a91b5b admin admin] The token cache is being invalidate because project 5a6986baffaf4f3a842ea1fe2a974741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:47:20.673128 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ba19c363-c78e-4147-9efb-851735a91b5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:20.673793 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-ba19c363-c78e-4147-9efb-851735a91b5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:20.675346 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1406/2829] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:47:20 2022] DELETE /identity/v3/projects/5a6986baffaf4f3a842ea1fe2a974741 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:47:30.325789 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=26,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:47:30.669977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=73,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:47:46.359798 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31e367-8982-48a2-ab1d-c0017e25f5f5 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:46.361354 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ce31e367-8982-48a2-ab1d-c0017e25f5f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:46.432815 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31e367-8982-48a2-ab1d-c0017e25f5f5 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:46.491619 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015583c-29ac-473b-817e-e04631c09d3a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49325c1733174e0c98b3475a10c9b5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:46.492265 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015583c-29ac-473b-817e-e04631c09d3a service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:46.492553 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015583c-29ac-473b-817e-e04631c09d3a service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:46.492834 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015583c-29ac-473b-817e-e04631c09d3a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:46.494649 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-1015583c-29ac-473b-817e-e04631c09d3a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:46.506104 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015583c-29ac-473b-817e-e04631c09d3a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:46.507673 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015583c-29ac-473b-817e-e04631c09d3a service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:46.517985 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1424/2830] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:47:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:47:56.513836 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=58 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:47:56.881925 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba19c363-c78e-4147-9efb-851735a91b5b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:56.883331 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-ba19c363-c78e-4147-9efb-851735a91b5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:56.947771 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba19c363-c78e-4147-9efb-851735a91b5b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:56.953041 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cf3b8-26df-4612-8759-5fcf94bc88c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:56.954006 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cf3b8-26df-4612-8759-5fcf94bc88c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:56.954320 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cf3b8-26df-4612-8759-5fcf94bc88c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:56.954660 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cf3b8-26df-4612-8759-5fcf94bc88c4 admin admin] PATH_INFO: `/v3/users/b82f5ce61ad742f888cd89a07e7cdb91` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:56.955904 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8cf3b8-26df-4612-8759-5fcf94bc88c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b82f5ce61ad742f888cd89a07e7cdb91)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:56.957874 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8cf3b8-26df-4612-8759-5fcf94bc88c4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:57.034213 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3e8cf3b8-26df-4612-8759-5fcf94bc88c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b82f5ce61ad742f888cd89a07e7cdb91'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:57.036116 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3e8cf3b8-26df-4612-8759-5fcf94bc88c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b82f5ce61ad742f888cd89a07e7cdb91'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:57.048859 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3e8cf3b8-26df-4612-8759-5fcf94bc88c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b82f5ce61ad742f888cd89a07e7cdb91'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:57.056360 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3e8cf3b8-26df-4612-8759-5fcf94bc88c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b82f5ce61ad742f888cd89a07e7cdb91'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:57.057026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-3e8cf3b8-26df-4612-8759-5fcf94bc88c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b82f5ce61ad742f888cd89a07e7cdb91'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:57.070541 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1407/2831] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:47:56 2022] DELETE /identity/v3/users/b82f5ce61ad742f888cd89a07e7cdb91 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:47:57.075844 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85574f-fdae-4bff-b547-833a80c1a32e None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:57.076142 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85574f-fdae-4bff-b547-833a80c1a32e None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:57.076429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85574f-fdae-4bff-b547-833a80c1a32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:57.093140 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-6e85574f-fdae-4bff-b547-833a80c1a32e None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:47:57.094385 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6e85574f-fdae-4bff-b547-833a80c1a32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:57.152520 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1425/2832] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:47:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:47:57.162178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cf3b8-26df-4612-8759-5fcf94bc88c4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:57.163580 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3e8cf3b8-26df-4612-8759-5fcf94bc88c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:57.220333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cf3b8-26df-4612-8759-5fcf94bc88c4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:57.223930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5ff0b-b498-4ec0-b6eb-2435444014f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:57.224567 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5ff0b-b498-4ec0-b6eb-2435444014f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:57.224848 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5ff0b-b498-4ec0-b6eb-2435444014f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:57.225120 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5ff0b-b498-4ec0-b6eb-2435444014f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:57.234809 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb5ff0b-b498-4ec0-b6eb-2435444014f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:57.236232 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb5ff0b-b498-4ec0-b6eb-2435444014f1 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:57.245510 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1408/2833] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:47:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:47:57.522907 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85574f-fdae-4bff-b547-833a80c1a32e None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:57.524992 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-6e85574f-fdae-4bff-b547-833a80c1a32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:57.537602 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85574f-fdae-4bff-b547-833a80c1a32e None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:57.542462 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa3a36-9782-4aae-ba21-119a29bdf131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:57.543357 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa3a36-9782-4aae-ba21-119a29bdf131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:57.543794 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa3a36-9782-4aae-ba21-119a29bdf131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:57.544017 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa3a36-9782-4aae-ba21-119a29bdf131 admin admin] PATH_INFO: `/v3/projects/59e9084b8f704da2be0721b548dfb781` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:57.545246 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa3a36-9782-4aae-ba21-119a29bdf131 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59e9084b8f704da2be0721b548dfb781)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:57.547203 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa3a36-9782-4aae-ba21-119a29bdf131 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:57.594798 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b2aa3a36-9782-4aae-ba21-119a29bdf131 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59e9084b8f704da2be0721b548dfb781', 'request_id': 'req-b2aa3a36-9782-4aae-ba21-119a29bdf131'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:57.600948 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b2aa3a36-9782-4aae-ba21-119a29bdf131 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59e9084b8f704da2be0721b548dfb781', 'request_id': 'req-b2aa3a36-9782-4aae-ba21-119a29bdf131'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:57.606979 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b2aa3a36-9782-4aae-ba21-119a29bdf131 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59e9084b8f704da2be0721b548dfb781', 'request_id': 'req-b2aa3a36-9782-4aae-ba21-119a29bdf131'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:57.619080 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b2aa3a36-9782-4aae-ba21-119a29bdf131 admin admin] The token cache is being invalidate because project 59e9084b8f704da2be0721b548dfb781 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:47:57.619374 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b2aa3a36-9782-4aae-ba21-119a29bdf131 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:57.620031 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-b2aa3a36-9782-4aae-ba21-119a29bdf131 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:57.621773 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1426/2834] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:47:57 2022] DELETE /identity/v3/projects/59e9084b8f704da2be0721b548dfb781 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:47:57.625832 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5ff0b-b498-4ec0-b6eb-2435444014f1 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:57.637738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5ff0b-b498-4ec0-b6eb-2435444014f1 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:57.656658 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:47:57.701162 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c5068-04f6-49ee-aced-a0a75d4430a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:57.702071 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c5068-04f6-49ee-aced-a0a75d4430a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:57.702358 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c5068-04f6-49ee-aced-a0a75d4430a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:57.702636 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c5068-04f6-49ee-aced-a0a75d4430a3 admin admin] PATH_INFO: `/v3/users/2bc5f4636f3542caba1dbf963d947a89` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:57.703831 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c5068-04f6-49ee-aced-a0a75d4430a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bc5f4636f3542caba1dbf963d947a89)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:57.705785 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c5068-04f6-49ee-aced-a0a75d4430a3 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:57.750336 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bd8c5068-04f6-49ee-aced-a0a75d4430a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bc5f4636f3542caba1dbf963d947a89'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:57.750981 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bd8c5068-04f6-49ee-aced-a0a75d4430a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bc5f4636f3542caba1dbf963d947a89'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:57.763206 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bd8c5068-04f6-49ee-aced-a0a75d4430a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bc5f4636f3542caba1dbf963d947a89'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:57.771042 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bd8c5068-04f6-49ee-aced-a0a75d4430a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bc5f4636f3542caba1dbf963d947a89'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:57.771722 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bd8c5068-04f6-49ee-aced-a0a75d4430a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bc5f4636f3542caba1dbf963d947a89'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:57.785457 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1409/2835] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:47:57 2022] DELETE /identity/v3/users/2bc5f4636f3542caba1dbf963d947a89 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:47:58.063605 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa3a36-9782-4aae-ba21-119a29bdf131 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:58.065335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b2aa3a36-9782-4aae-ba21-119a29bdf131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:58.081188 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa3a36-9782-4aae-ba21-119a29bdf131 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:58.139840 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e1312-60f2-4bdd-94a6-415561fbe049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:58.140629 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e1312-60f2-4bdd-94a6-415561fbe049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:58.140916 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e1312-60f2-4bdd-94a6-415561fbe049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:58.141199 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e1312-60f2-4bdd-94a6-415561fbe049 admin admin] PATH_INFO: `/v3/projects/dc533bf24c584bac872ada7bdc8e9076` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:58.142424 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e1312-60f2-4bdd-94a6-415561fbe049 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc533bf24c584bac872ada7bdc8e9076)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:58.144351 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e1312-60f2-4bdd-94a6-415561fbe049 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:58.174403 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:47:58.192639 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-317e1312-60f2-4bdd-94a6-415561fbe049 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc533bf24c584bac872ada7bdc8e9076', 'request_id': 'req-317e1312-60f2-4bdd-94a6-415561fbe049'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:58.198874 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-317e1312-60f2-4bdd-94a6-415561fbe049 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc533bf24c584bac872ada7bdc8e9076', 'request_id': 'req-317e1312-60f2-4bdd-94a6-415561fbe049'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:58.204466 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-317e1312-60f2-4bdd-94a6-415561fbe049 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc533bf24c584bac872ada7bdc8e9076', 'request_id': 'req-317e1312-60f2-4bdd-94a6-415561fbe049'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:58.216890 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-317e1312-60f2-4bdd-94a6-415561fbe049 admin admin] The token cache is being invalidate because project dc533bf24c584bac872ada7bdc8e9076 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:47:58.217187 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-317e1312-60f2-4bdd-94a6-415561fbe049 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:58.217850 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-317e1312-60f2-4bdd-94a6-415561fbe049 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:47:58.219455 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1427/2836] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:47:58 2022] DELETE /identity/v3/projects/dc533bf24c584bac872ada7bdc8e9076 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:47:58.224593 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8868c-2ac0-4c72-be19-2dbf89ef2c93 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:58.225113 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8868c-2ac0-4c72-be19-2dbf89ef2c93 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:58.225232 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8868c-2ac0-4c72-be19-2dbf89ef2c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:58.243233 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-9fd8868c-2ac0-4c72-be19-2dbf89ef2c93 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:47:58.244432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-9fd8868c-2ac0-4c72-be19-2dbf89ef2c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:58.303560 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1410/2837] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:47:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:47:58.361254 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587544b-f0e9-45fb-acc2-6abc1f3438fd None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:58.361545 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587544b-f0e9-45fb-acc2-6abc1f3438fd None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:58.361866 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587544b-f0e9-45fb-acc2-6abc1f3438fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:58.380198 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-4587544b-f0e9-45fb-acc2-6abc1f3438fd None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:47:58.381395 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-4587544b-f0e9-45fb-acc2-6abc1f3438fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:58.389780 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1428/2838] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:47:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:47:58.394221 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8868c-2ac0-4c72-be19-2dbf89ef2c93 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:58.404741 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8868c-2ac0-4c72-be19-2dbf89ef2c93 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:58.409284 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bbda0-0e89-44dd-a9e0-7999f8a88ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:58.409969 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bbda0-0e89-44dd-a9e0-7999f8a88ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:58.410255 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bbda0-0e89-44dd-a9e0-7999f8a88ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:58.410534 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bbda0-0e89-44dd-a9e0-7999f8a88ed2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:58.411250 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940bbda0-0e89-44dd-a9e0-7999f8a88ed2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:47:58.411555 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940bbda0-0e89-44dd-a9e0-7999f8a88ed2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:58.413146 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940bbda0-0e89-44dd-a9e0-7999f8a88ed2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:58.424789 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1411/2839] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:47:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:47:58.429084 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587544b-f0e9-45fb-acc2-6abc1f3438fd None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:58.438452 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587544b-f0e9-45fb-acc2-6abc1f3438fd None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:58.443122 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9c5a6-7a13-4f68-b2af-2bed07d6cd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:58.443771 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9c5a6-7a13-4f68-b2af-2bed07d6cd3f admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:58.444060 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9c5a6-7a13-4f68-b2af-2bed07d6cd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:58.444345 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9c5a6-7a13-4f68-b2af-2bed07d6cd3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:58.445032 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff9c5a6-7a13-4f68-b2af-2bed07d6cd3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:58.446996 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff9c5a6-7a13-4f68-b2af-2bed07d6cd3f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:58.473538 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1429/2840] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:47:58 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:47:58.477617 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bbda0-0e89-44dd-a9e0-7999f8a88ed2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:58.487553 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bbda0-0e89-44dd-a9e0-7999f8a88ed2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:58.513648 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:47:58.545843 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1cb6d1-f1ac-482c-b03b-db530448efd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:58.546497 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cb6d1-f1ac-482c-b03b-db530448efd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:58.546779 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cb6d1-f1ac-482c-b03b-db530448efd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:58.547054 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cb6d1-f1ac-482c-b03b-db530448efd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:58.547732 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1cb6d1-f1ac-482c-b03b-db530448efd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:58.549668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1cb6d1-f1ac-482c-b03b-db530448efd8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:58.568869 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1412/2841] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:47:58 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:47:58.573004 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9c5a6-7a13-4f68-b2af-2bed07d6cd3f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:58.583975 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9c5a6-7a13-4f68-b2af-2bed07d6cd3f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:58.588797 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd455e5-cacf-45a8-8bb6-1479d6bc26f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:58.589612 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd455e5-cacf-45a8-8bb6-1479d6bc26f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:58.590099 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd455e5-cacf-45a8-8bb6-1479d6bc26f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:58.590455 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd455e5-cacf-45a8-8bb6-1479d6bc26f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:58.591547 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd455e5-cacf-45a8-8bb6-1479d6bc26f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:58.594055 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd455e5-cacf-45a8-8bb6-1479d6bc26f3 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:58.604409 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1430/2842] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:47:58.608397 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1cb6d1-f1ac-482c-b03b-db530448efd8 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:58.618280 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1cb6d1-f1ac-482c-b03b-db530448efd8 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:58.623565 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e7dd3-0f2c-4d15-bde5-dda1905cb6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:58.624207 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e7dd3-0f2c-4d15-bde5-dda1905cb6ec admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:58.624490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e7dd3-0f2c-4d15-bde5-dda1905cb6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:58.624766 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e7dd3-0f2c-4d15-bde5-dda1905cb6ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:58.625485 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e7dd3-0f2c-4d15-bde5-dda1905cb6ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:58.627111 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e7dd3-0f2c-4d15-bde5-dda1905cb6ec admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:58.636750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1413/2843] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:47:58.642038 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd455e5-cacf-45a8-8bb6-1479d6bc26f3 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:58.651837 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd455e5-cacf-45a8-8bb6-1479d6bc26f3 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:58.656736 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562d0cd-96d3-485f-bf77-abd2cbe5e30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:58.657760 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562d0cd-96d3-485f-bf77-abd2cbe5e30c admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:58.658044 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562d0cd-96d3-485f-bf77-abd2cbe5e30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:58.658323 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562d0cd-96d3-485f-bf77-abd2cbe5e30c admin admin] PATH_INFO: `/v3/projects/a3ab80c14d2c4ee8987bc5fa1ee49e08/users/41dff29b2f2f45ed8c30460bd4692f97/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:58.672786 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562d0cd-96d3-485f-bf77-abd2cbe5e30c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3ab80c14d2c4ee8987bc5fa1ee49e08, user_id=41dff29b2f2f45ed8c30460bd4692f97, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:58.675454 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562d0cd-96d3-485f-bf77-abd2cbe5e30c admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:58.688841 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1431/2844] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:47:58 2022] PUT /identity/v3/projects/a3ab80c14d2c4ee8987bc5fa1ee49e08/users/41dff29b2f2f45ed8c30460bd4692f97/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:47:58.698785 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b5f72-9c78-4fc9-9d35-a5a44a1ca837 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:58.699080 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b5f72-9c78-4fc9-9d35-a5a44a1ca837 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:58.699390 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b5f72-9c78-4fc9-9d35-a5a44a1ca837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:58.725678 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-587b5f72-9c78-4fc9-9d35-a5a44a1ca837 None None] MFA Rules not processed for user `41dff29b2f2f45ed8c30460bd4692f97`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:47:58.726873 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-587b5f72-9c78-4fc9-9d35-a5a44a1ca837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:58.791388 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1414/2845] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:47:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:47:58.796211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562d0cd-96d3-485f-bf77-abd2cbe5e30c admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:58.857996 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562d0cd-96d3-485f-bf77-abd2cbe5e30c admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:58.862692 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8278404b-9a26-4579-9bd6-7a7dd2650751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:58.863339 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278404b-9a26-4579-9bd6-7a7dd2650751 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:58.863628 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278404b-9a26-4579-9bd6-7a7dd2650751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:58.863904 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278404b-9a26-4579-9bd6-7a7dd2650751 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:58.864577 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8278404b-9a26-4579-9bd6-7a7dd2650751 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:58.866531 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8278404b-9a26-4579-9bd6-7a7dd2650751 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:58.892051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1432/2846] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:47:58 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:47:58.896234 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b5f72-9c78-4fc9-9d35-a5a44a1ca837 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:58.907135 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b5f72-9c78-4fc9-9d35-a5a44a1ca837 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:58.962674 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7178d1a-a38b-4ca4-8704-1dcf0be75f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:58.963327 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7178d1a-a38b-4ca4-8704-1dcf0be75f81 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:58.963615 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7178d1a-a38b-4ca4-8704-1dcf0be75f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:58.963889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7178d1a-a38b-4ca4-8704-1dcf0be75f81 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:58.964570 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7178d1a-a38b-4ca4-8704-1dcf0be75f81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:58.966536 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7178d1a-a38b-4ca4-8704-1dcf0be75f81 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:58.986198 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1415/2847] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:47:58 2022] POST /identity/v3/users => generated 296 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:47:58.990252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8278404b-9a26-4579-9bd6-7a7dd2650751 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:59.001047 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8278404b-9a26-4579-9bd6-7a7dd2650751 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:59.005526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eee60c-91d8-47fa-825e-f5e3708006d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:59.006214 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eee60c-91d8-47fa-825e-f5e3708006d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:59.006497 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eee60c-91d8-47fa-825e-f5e3708006d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:59.006773 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eee60c-91d8-47fa-825e-f5e3708006d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:59.007484 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eee60c-91d8-47fa-825e-f5e3708006d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:59.009059 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eee60c-91d8-47fa-825e-f5e3708006d6 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:59.019563 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1433/2848] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:47:59.023681 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7178d1a-a38b-4ca4-8704-1dcf0be75f81 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:59.033708 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7178d1a-a38b-4ca4-8704-1dcf0be75f81 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:59.038285 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345b1f7-2415-49c4-a313-7ae9ae76799a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:59.039200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345b1f7-2415-49c4-a313-7ae9ae76799a admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:59.039483 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345b1f7-2415-49c4-a313-7ae9ae76799a admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:59.039761 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345b1f7-2415-49c4-a313-7ae9ae76799a admin admin] PATH_INFO: `/v3/projects/bcff66ff09ab40f597ff09e0da606355/users/8ebb26c8b4474a5d8f5249c9df29183d/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:59.056422 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345b1f7-2415-49c4-a313-7ae9ae76799a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcff66ff09ab40f597ff09e0da606355, user_id=8ebb26c8b4474a5d8f5249c9df29183d, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:59.060379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345b1f7-2415-49c4-a313-7ae9ae76799a admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:59.077078 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1416/2849] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:47:59 2022] PUT /identity/v3/projects/bcff66ff09ab40f597ff09e0da606355/users/8ebb26c8b4474a5d8f5249c9df29183d/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:47:59.080966 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eee60c-91d8-47fa-825e-f5e3708006d6 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:59.090889 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eee60c-91d8-47fa-825e-f5e3708006d6 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:59.111076 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:47:59.149089 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ecbb27-061d-4e3c-96f5-d87fe8d84989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:59.149800 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ecbb27-061d-4e3c-96f5-d87fe8d84989 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:59.150107 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ecbb27-061d-4e3c-96f5-d87fe8d84989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:59.150383 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ecbb27-061d-4e3c-96f5-d87fe8d84989 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:59.151092 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ecbb27-061d-4e3c-96f5-d87fe8d84989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:59.152706 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ecbb27-061d-4e3c-96f5-d87fe8d84989 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:59.162851 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1434/2850] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:47:59.167058 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345b1f7-2415-49c4-a313-7ae9ae76799a admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:59.173535 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:47:59.179941 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345b1f7-2415-49c4-a313-7ae9ae76799a admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:59.185534 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c1fca-37db-4831-9926-f6b7cee8ec83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:59.186611 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c1fca-37db-4831-9926-f6b7cee8ec83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:59.186900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c1fca-37db-4831-9926-f6b7cee8ec83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:59.187177 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c1fca-37db-4831-9926-f6b7cee8ec83 admin admin] PATH_INFO: `/v3/projects/bcff66ff09ab40f597ff09e0da606355/users/8ebb26c8b4474a5d8f5249c9df29183d/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:59.189185 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c1fca-37db-4831-9926-f6b7cee8ec83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcff66ff09ab40f597ff09e0da606355, user_id=8ebb26c8b4474a5d8f5249c9df29183d, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:59.191973 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c1fca-37db-4831-9926-f6b7cee8ec83 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:59.202314 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1417/2851] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:47:59 2022] PUT /identity/v3/projects/bcff66ff09ab40f597ff09e0da606355/users/8ebb26c8b4474a5d8f5249c9df29183d/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:47:59.212330 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19c438-a73c-4050-b0d6-89264c459abe None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:59.212629 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19c438-a73c-4050-b0d6-89264c459abe None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:59.212917 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19c438-a73c-4050-b0d6-89264c459abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:59.239230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-2f19c438-a73c-4050-b0d6-89264c459abe None None] MFA Rules not processed for user `8ebb26c8b4474a5d8f5249c9df29183d`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:47:59.240443 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-2f19c438-a73c-4050-b0d6-89264c459abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:59.308915 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1435/2852] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:47:59 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:47:59.320220 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c1fca-37db-4831-9926-f6b7cee8ec83 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:59.321926 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-db8c1fca-37db-4831-9926-f6b7cee8ec83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:47:59.378242 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c1fca-37db-4831-9926-f6b7cee8ec83 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:59.381951 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fea43-f34c-480b-9b79-6d1dbe587ba3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:59.382578 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fea43-f34c-480b-9b79-6d1dbe587ba3 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:59.382858 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fea43-f34c-480b-9b79-6d1dbe587ba3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:59.383133 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fea43-f34c-480b-9b79-6d1dbe587ba3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:59.392282 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3fea43-f34c-480b-9b79-6d1dbe587ba3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:59.393737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3fea43-f34c-480b-9b79-6d1dbe587ba3 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:59.402928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1418/2853] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:47:59 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:47:59.475260 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f19c438-a73c-4050-b0d6-89264c459abe None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:47:59.486258 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f19c438-a73c-4050-b0d6-89264c459abe None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:47:59.489918 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d7e6c8-d490-42ef-9924-6be6aaa8165f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:47:59.490548 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7e6c8-d490-42ef-9924-6be6aaa8165f service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:47:59.490834 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7e6c8-d490-42ef-9924-6be6aaa8165f service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:47:59.491114 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7e6c8-d490-42ef-9924-6be6aaa8165f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:47:59.549540 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d7e6c8-d490-42ef-9924-6be6aaa8165f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:47:59.551038 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d7e6c8-d490-42ef-9924-6be6aaa8165f service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:47:59.560293 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1436/2854] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:47:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:48:09.398773 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:48:09.500922 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fea43-f34c-480b-9b79-6d1dbe587ba3 service cinder] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:09.555997 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=80 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:48:09.563005 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fea43-f34c-480b-9b79-6d1dbe587ba3 service cinder] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:09.567540 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45399a3a-a3da-4353-8bac-654ebc1b878e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:09.568393 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45399a3a-a3da-4353-8bac-654ebc1b878e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:09.568676 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45399a3a-a3da-4353-8bac-654ebc1b878e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:09.568953 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45399a3a-a3da-4353-8bac-654ebc1b878e admin admin] PATH_INFO: `/v3/users/41dff29b2f2f45ed8c30460bd4692f97` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:09.570153 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45399a3a-a3da-4353-8bac-654ebc1b878e admin admin] RBAC: Authorizing `identity:delete_user(user_id=41dff29b2f2f45ed8c30460bd4692f97)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:09.572072 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45399a3a-a3da-4353-8bac-654ebc1b878e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:09.618228 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-45399a3a-a3da-4353-8bac-654ebc1b878e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41dff29b2f2f45ed8c30460bd4692f97'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:09.618888 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-45399a3a-a3da-4353-8bac-654ebc1b878e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41dff29b2f2f45ed8c30460bd4692f97'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:09.631754 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-45399a3a-a3da-4353-8bac-654ebc1b878e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41dff29b2f2f45ed8c30460bd4692f97'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:09.640892 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-45399a3a-a3da-4353-8bac-654ebc1b878e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41dff29b2f2f45ed8c30460bd4692f97'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:09.641494 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-45399a3a-a3da-4353-8bac-654ebc1b878e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41dff29b2f2f45ed8c30460bd4692f97'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:09.655627 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1419/2855] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:48:09 2022] DELETE /identity/v3/users/41dff29b2f2f45ed8c30460bd4692f97 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:48:09.660350 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85810402-2f9f-438c-affc-8f7e673b42c9 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:09.660639 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85810402-2f9f-438c-affc-8f7e673b42c9 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:09.660920 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85810402-2f9f-438c-affc-8f7e673b42c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:09.679203 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-85810402-2f9f-438c-affc-8f7e673b42c9 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:48:09.680429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-85810402-2f9f-438c-affc-8f7e673b42c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:09.739731 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1437/2856] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:48:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:48:09.750244 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45399a3a-a3da-4353-8bac-654ebc1b878e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:09.751635 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-45399a3a-a3da-4353-8bac-654ebc1b878e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:09.809749 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45399a3a-a3da-4353-8bac-654ebc1b878e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:09.813803 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59bab0f-7b62-4d9d-b887-0dc87b4bd1b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:09.814440 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59bab0f-7b62-4d9d-b887-0dc87b4bd1b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:09.814727 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59bab0f-7b62-4d9d-b887-0dc87b4bd1b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:09.815008 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59bab0f-7b62-4d9d-b887-0dc87b4bd1b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:09.823979 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59bab0f-7b62-4d9d-b887-0dc87b4bd1b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:09.825415 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59bab0f-7b62-4d9d-b887-0dc87b4bd1b2 service neutron] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:09.835399 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1420/2857] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:48:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:48:10.112504 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85810402-2f9f-438c-affc-8f7e673b42c9 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:10.113990 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-85810402-2f9f-438c-affc-8f7e673b42c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:10.128807 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85810402-2f9f-438c-affc-8f7e673b42c9 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:10.133546 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b3cd8e-8a08-4871-89ab-7a2e2702dfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:10.134509 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3cd8e-8a08-4871-89ab-7a2e2702dfc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:10.134791 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3cd8e-8a08-4871-89ab-7a2e2702dfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:10.135066 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3cd8e-8a08-4871-89ab-7a2e2702dfc8 admin admin] PATH_INFO: `/v3/projects/a3ab80c14d2c4ee8987bc5fa1ee49e08` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:10.136277 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b3cd8e-8a08-4871-89ab-7a2e2702dfc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3ab80c14d2c4ee8987bc5fa1ee49e08)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:10.138295 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b3cd8e-8a08-4871-89ab-7a2e2702dfc8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:10.185019 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d0b3cd8e-8a08-4871-89ab-7a2e2702dfc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3ab80c14d2c4ee8987bc5fa1ee49e08', 'request_id': 'req-d0b3cd8e-8a08-4871-89ab-7a2e2702dfc8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:10.191651 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d0b3cd8e-8a08-4871-89ab-7a2e2702dfc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3ab80c14d2c4ee8987bc5fa1ee49e08', 'request_id': 'req-d0b3cd8e-8a08-4871-89ab-7a2e2702dfc8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:10.197314 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d0b3cd8e-8a08-4871-89ab-7a2e2702dfc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3ab80c14d2c4ee8987bc5fa1ee49e08', 'request_id': 'req-d0b3cd8e-8a08-4871-89ab-7a2e2702dfc8'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:10.210444 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d0b3cd8e-8a08-4871-89ab-7a2e2702dfc8 admin admin] The token cache is being invalidate because project a3ab80c14d2c4ee8987bc5fa1ee49e08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:48:10.210735 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d0b3cd8e-8a08-4871-89ab-7a2e2702dfc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:10.211371 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d0b3cd8e-8a08-4871-89ab-7a2e2702dfc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:10.213233 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1438/2858] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:48:10 2022] DELETE /identity/v3/projects/a3ab80c14d2c4ee8987bc5fa1ee49e08 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:48:10.217948 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59bab0f-7b62-4d9d-b887-0dc87b4bd1b2 service neutron] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:10.227575 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59bab0f-7b62-4d9d-b887-0dc87b4bd1b2 service neutron] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:10.246553 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:48:10.289523 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4d8ed-d94f-4c12-884b-1899328e0498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:10.290428 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4d8ed-d94f-4c12-884b-1899328e0498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:10.290710 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4d8ed-d94f-4c12-884b-1899328e0498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:10.290992 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4d8ed-d94f-4c12-884b-1899328e0498 admin admin] PATH_INFO: `/v3/users/8ebb26c8b4474a5d8f5249c9df29183d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:10.292437 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4d8ed-d94f-4c12-884b-1899328e0498 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ebb26c8b4474a5d8f5249c9df29183d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:10.294409 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4d8ed-d94f-4c12-884b-1899328e0498 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:10.340977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-45e4d8ed-d94f-4c12-884b-1899328e0498 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ebb26c8b4474a5d8f5249c9df29183d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:10.342470 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-45e4d8ed-d94f-4c12-884b-1899328e0498 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ebb26c8b4474a5d8f5249c9df29183d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:10.354333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-45e4d8ed-d94f-4c12-884b-1899328e0498 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ebb26c8b4474a5d8f5249c9df29183d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:10.362054 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-45e4d8ed-d94f-4c12-884b-1899328e0498 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ebb26c8b4474a5d8f5249c9df29183d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:10.362613 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-45e4d8ed-d94f-4c12-884b-1899328e0498 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ebb26c8b4474a5d8f5249c9df29183d'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:10.374820 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1421/2859] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:48:10 2022] DELETE /identity/v3/users/8ebb26c8b4474a5d8f5249c9df29183d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:48:10.650986 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b3cd8e-8a08-4871-89ab-7a2e2702dfc8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:10.652476 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-d0b3cd8e-8a08-4871-89ab-7a2e2702dfc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:10.669064 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b3cd8e-8a08-4871-89ab-7a2e2702dfc8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:10.729196 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4afda1b-8850-4cd8-80bf-c4fc29f919b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:10.730122 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afda1b-8850-4cd8-80bf-c4fc29f919b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:10.730427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afda1b-8850-4cd8-80bf-c4fc29f919b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:10.730704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afda1b-8850-4cd8-80bf-c4fc29f919b5 admin admin] PATH_INFO: `/v3/projects/bcff66ff09ab40f597ff09e0da606355` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:10.731872 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4afda1b-8850-4cd8-80bf-c4fc29f919b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcff66ff09ab40f597ff09e0da606355)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:10.733872 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4afda1b-8850-4cd8-80bf-c4fc29f919b5 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:10.763117 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:48:10.780034 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c4afda1b-8850-4cd8-80bf-c4fc29f919b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcff66ff09ab40f597ff09e0da606355', 'request_id': 'req-c4afda1b-8850-4cd8-80bf-c4fc29f919b5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:10.785970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c4afda1b-8850-4cd8-80bf-c4fc29f919b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcff66ff09ab40f597ff09e0da606355', 'request_id': 'req-c4afda1b-8850-4cd8-80bf-c4fc29f919b5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:10.791414 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c4afda1b-8850-4cd8-80bf-c4fc29f919b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcff66ff09ab40f597ff09e0da606355', 'request_id': 'req-c4afda1b-8850-4cd8-80bf-c4fc29f919b5'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:10.803713 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c4afda1b-8850-4cd8-80bf-c4fc29f919b5 admin admin] The token cache is being invalidate because project bcff66ff09ab40f597ff09e0da606355 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80207) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:48:10.803986 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c4afda1b-8850-4cd8-80bf-c4fc29f919b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:10.804598 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-c4afda1b-8850-4cd8-80bf-c4fc29f919b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:10.806162 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1439/2860] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:48:10 2022] DELETE /identity/v3/projects/bcff66ff09ab40f597ff09e0da606355 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:48:10.811709 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd839f-ebf4-436c-99f7-24398204c32d None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:10.812191 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd839f-ebf4-436c-99f7-24398204c32d None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:10.812305 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd839f-ebf4-436c-99f7-24398204c32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:10.830009 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-cecd839f-ebf4-436c-99f7-24398204c32d None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:48:10.831200 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-cecd839f-ebf4-436c-99f7-24398204c32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:10.889464 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1422/2861] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:48:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:48:10.897382 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40fc2b-02bc-4fd5-b8fc-8c5e4406d31b None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:10.897637 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40fc2b-02bc-4fd5-b8fc-8c5e4406d31b None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:10.897923 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40fc2b-02bc-4fd5-b8fc-8c5e4406d31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:10.916003 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-fc40fc2b-02bc-4fd5-b8fc-8c5e4406d31b None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:48:10.917434 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-fc40fc2b-02bc-4fd5-b8fc-8c5e4406d31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:10.925950 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1440/2862] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:48:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:48:10.930092 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd839f-ebf4-436c-99f7-24398204c32d None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:10.940867 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd839f-ebf4-436c-99f7-24398204c32d None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:10.945501 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b88aae3-81f6-4ab6-89c1-3b809254aa3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:10.946178 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88aae3-81f6-4ab6-89c1-3b809254aa3c admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:10.946492 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88aae3-81f6-4ab6-89c1-3b809254aa3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:10.946768 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88aae3-81f6-4ab6-89c1-3b809254aa3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:10.947489 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b88aae3-81f6-4ab6-89c1-3b809254aa3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80206) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:48:10.947823 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b88aae3-81f6-4ab6-89c1-3b809254aa3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:10.949419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b88aae3-81f6-4ab6-89c1-3b809254aa3c admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:10.961188 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1423/2863] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:48:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:48:10.965499 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40fc2b-02bc-4fd5-b8fc-8c5e4406d31b None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:10.974879 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40fc2b-02bc-4fd5-b8fc-8c5e4406d31b None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:10.979640 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38554261-9831-4bec-a919-981b99f684c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:10.980283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38554261-9831-4bec-a919-981b99f684c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:10.980572 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38554261-9831-4bec-a919-981b99f684c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:10.980856 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38554261-9831-4bec-a919-981b99f684c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:10.981559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38554261-9831-4bec-a919-981b99f684c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:10.984125 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38554261-9831-4bec-a919-981b99f684c0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:11.010880 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1441/2864] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:48:10 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:48:11.015037 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b88aae3-81f6-4ab6-89c1-3b809254aa3c admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:11.024902 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b88aae3-81f6-4ab6-89c1-3b809254aa3c admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:11.051217 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:48:11.083499 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beeab33-e0db-4fe2-a362-31fab66c6177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:11.084163 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beeab33-e0db-4fe2-a362-31fab66c6177 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:11.084450 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beeab33-e0db-4fe2-a362-31fab66c6177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:11.084728 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beeab33-e0db-4fe2-a362-31fab66c6177 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:11.085423 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beeab33-e0db-4fe2-a362-31fab66c6177 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:11.087387 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beeab33-e0db-4fe2-a362-31fab66c6177 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:11.106750 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1424/2865] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:48:11 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:48:11.111402 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38554261-9831-4bec-a919-981b99f684c0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:11.121814 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38554261-9831-4bec-a919-981b99f684c0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:11.126893 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f472c81b-0d20-4598-bd8a-dac24b1ea936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:11.127550 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472c81b-0d20-4598-bd8a-dac24b1ea936 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:11.127839 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472c81b-0d20-4598-bd8a-dac24b1ea936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:11.128123 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472c81b-0d20-4598-bd8a-dac24b1ea936 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:11.128872 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f472c81b-0d20-4598-bd8a-dac24b1ea936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:11.130538 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f472c81b-0d20-4598-bd8a-dac24b1ea936 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:11.141557 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1442/2866] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:48:11.145419 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beeab33-e0db-4fe2-a362-31fab66c6177 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:11.156148 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beeab33-e0db-4fe2-a362-31fab66c6177 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:11.160976 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b30666-576d-4a3f-a401-1a9c500fe100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:11.161637 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b30666-576d-4a3f-a401-1a9c500fe100 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:11.161928 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b30666-576d-4a3f-a401-1a9c500fe100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:11.162215 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b30666-576d-4a3f-a401-1a9c500fe100 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:11.162945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b30666-576d-4a3f-a401-1a9c500fe100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:11.164527 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b30666-576d-4a3f-a401-1a9c500fe100 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:11.175292 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1425/2867] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:48:11.179290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f472c81b-0d20-4598-bd8a-dac24b1ea936 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:11.189936 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f472c81b-0d20-4598-bd8a-dac24b1ea936 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:11.194559 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283259eb-1e92-4a54-9302-58d6c98a9102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:11.195429 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283259eb-1e92-4a54-9302-58d6c98a9102 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:11.195716 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283259eb-1e92-4a54-9302-58d6c98a9102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:11.196000 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283259eb-1e92-4a54-9302-58d6c98a9102 admin admin] PATH_INFO: `/v3/projects/61d12ec947a44a108f8eeab409e3783e/users/70950b4927b14801b6af87eddb71dfa4/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:11.212368 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283259eb-1e92-4a54-9302-58d6c98a9102 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61d12ec947a44a108f8eeab409e3783e, user_id=70950b4927b14801b6af87eddb71dfa4, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:11.215055 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283259eb-1e92-4a54-9302-58d6c98a9102 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:11.227896 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1443/2868] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:48:11 2022] PUT /identity/v3/projects/61d12ec947a44a108f8eeab409e3783e/users/70950b4927b14801b6af87eddb71dfa4/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:48:11.238930 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502a6b8-ba82-462d-9ba5-74d5760bc060 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:11.239224 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502a6b8-ba82-462d-9ba5-74d5760bc060 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:11.239507 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502a6b8-ba82-462d-9ba5-74d5760bc060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:11.267171 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-3502a6b8-ba82-462d-9ba5-74d5760bc060 None None] MFA Rules not processed for user `70950b4927b14801b6af87eddb71dfa4`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:48:11.268406 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3502a6b8-ba82-462d-9ba5-74d5760bc060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:11.336114 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1426/2869] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:48:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:48:11.340646 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283259eb-1e92-4a54-9302-58d6c98a9102 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:11.405731 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283259eb-1e92-4a54-9302-58d6c98a9102 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:11.410629 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43eebe-e815-4d9d-b18c-223f1dab3df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:11.411261 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43eebe-e815-4d9d-b18c-223f1dab3df1 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:11.411554 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43eebe-e815-4d9d-b18c-223f1dab3df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:11.411838 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43eebe-e815-4d9d-b18c-223f1dab3df1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:11.412542 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43eebe-e815-4d9d-b18c-223f1dab3df1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:11.414597 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43eebe-e815-4d9d-b18c-223f1dab3df1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:11.439791 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1444/2870] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:48:11 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:48:11.443954 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502a6b8-ba82-462d-9ba5-74d5760bc060 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:11.455057 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502a6b8-ba82-462d-9ba5-74d5760bc060 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:11.512322 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c62f23-48af-4c81-a47c-8f09cc405993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:11.512987 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c62f23-48af-4c81-a47c-8f09cc405993 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:11.513276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c62f23-48af-4c81-a47c-8f09cc405993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:11.513556 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c62f23-48af-4c81-a47c-8f09cc405993 admin admin] PATH_INFO: `/v3/users` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:11.514281 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c62f23-48af-4c81-a47c-8f09cc405993 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:11.516211 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c62f23-48af-4c81-a47c-8f09cc405993 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:11.535791 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1427/2871] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:48:11 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:48:11.539847 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43eebe-e815-4d9d-b18c-223f1dab3df1 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:11.550905 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43eebe-e815-4d9d-b18c-223f1dab3df1 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:11.555508 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39125fec-3096-4655-aff5-6a3b257b5011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:11.556156 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39125fec-3096-4655-aff5-6a3b257b5011 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:11.556440 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39125fec-3096-4655-aff5-6a3b257b5011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:11.556717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39125fec-3096-4655-aff5-6a3b257b5011 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:11.557431 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39125fec-3096-4655-aff5-6a3b257b5011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:11.559063 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39125fec-3096-4655-aff5-6a3b257b5011 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:11.570041 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1445/2872] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:48:11.574016 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c62f23-48af-4c81-a47c-8f09cc405993 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:11.584670 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c62f23-48af-4c81-a47c-8f09cc405993 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:11.589391 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732cdb8-2998-464b-83c0-6b213258f3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:11.590398 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732cdb8-2998-464b-83c0-6b213258f3b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:11.590721 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732cdb8-2998-464b-83c0-6b213258f3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:11.591068 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732cdb8-2998-464b-83c0-6b213258f3b2 admin admin] PATH_INFO: `/v3/projects/07fc67e82cfa42d4a5ede0102bcd481c/users/101c25f34eb449e6afee6dbb402cb23b/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:11.607481 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7732cdb8-2998-464b-83c0-6b213258f3b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07fc67e82cfa42d4a5ede0102bcd481c, user_id=101c25f34eb449e6afee6dbb402cb23b, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:11.610209 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7732cdb8-2998-464b-83c0-6b213258f3b2 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:11.622476 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1428/2873] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:48:11 2022] PUT /identity/v3/projects/07fc67e82cfa42d4a5ede0102bcd481c/users/101c25f34eb449e6afee6dbb402cb23b/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:48:11.627528 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39125fec-3096-4655-aff5-6a3b257b5011 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:11.637604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39125fec-3096-4655-aff5-6a3b257b5011 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:11.660204 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:48:11.704021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47bde1-7014-4a5c-9cb8-32d8add39636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:11.704699 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47bde1-7014-4a5c-9cb8-32d8add39636 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:11.704989 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47bde1-7014-4a5c-9cb8-32d8add39636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:11.705270 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47bde1-7014-4a5c-9cb8-32d8add39636 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:11.706050 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47bde1-7014-4a5c-9cb8-32d8add39636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:11.707661 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47bde1-7014-4a5c-9cb8-32d8add39636 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:11.717840 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1446/2874] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:48:11.722021 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732cdb8-2998-464b-83c0-6b213258f3b2 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:11.727455 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:48:11.735050 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732cdb8-2998-464b-83c0-6b213258f3b2 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:11.739885 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3ad1d-ea51-40f4-949a-caade2aa927e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:11.740745 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3ad1d-ea51-40f4-949a-caade2aa927e admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:11.741026 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3ad1d-ea51-40f4-949a-caade2aa927e admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:11.741299 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3ad1d-ea51-40f4-949a-caade2aa927e admin admin] PATH_INFO: `/v3/projects/07fc67e82cfa42d4a5ede0102bcd481c/users/101c25f34eb449e6afee6dbb402cb23b/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:11.743276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf3ad1d-ea51-40f4-949a-caade2aa927e admin admin] RBAC: Authorizing `identity:create_grant(project_id=07fc67e82cfa42d4a5ede0102bcd481c, user_id=101c25f34eb449e6afee6dbb402cb23b, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:11.746025 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf3ad1d-ea51-40f4-949a-caade2aa927e admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:11.757149 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1429/2875] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:48:11 2022] PUT /identity/v3/projects/07fc67e82cfa42d4a5ede0102bcd481c/users/101c25f34eb449e6afee6dbb402cb23b/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:48:11.768861 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a50b06-fdf4-4615-bfbc-2aaad5def854 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:11.769211 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a50b06-fdf4-4615-bfbc-2aaad5def854 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:11.769469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a50b06-fdf4-4615-bfbc-2aaad5def854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:11.796533 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-b2a50b06-fdf4-4615-bfbc-2aaad5def854 None None] MFA Rules not processed for user `101c25f34eb449e6afee6dbb402cb23b`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:48:11.797780 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-b2a50b06-fdf4-4615-bfbc-2aaad5def854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:11.865573 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1447/2876] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:48:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:48:11.877981 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3ad1d-ea51-40f4-949a-caade2aa927e admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:11.879694 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-3cf3ad1d-ea51-40f4-949a-caade2aa927e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:11.937185 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3ad1d-ea51-40f4-949a-caade2aa927e admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:11.940857 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd0ee3-1e04-4b4a-92c9-9734db2fe915 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:11.941489 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd0ee3-1e04-4b4a-92c9-9734db2fe915 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:11.941806 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd0ee3-1e04-4b4a-92c9-9734db2fe915 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:11.942089 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd0ee3-1e04-4b4a-92c9-9734db2fe915 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:11.951469 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd0ee3-1e04-4b4a-92c9-9734db2fe915 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:11.952889 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd0ee3-1e04-4b4a-92c9-9734db2fe915 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:11.962320 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1430/2877] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:48:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:48:12.030336 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a50b06-fdf4-4615-bfbc-2aaad5def854 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:12.093709 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a50b06-fdf4-4615-bfbc-2aaad5def854 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:12.098315 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede6b190-ea59-4590-ad82-ca1928960118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:12.099185 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6b190-ea59-4590-ad82-ca1928960118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:12.099469 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6b190-ea59-4590-ad82-ca1928960118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:12.099748 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6b190-ea59-4590-ad82-ca1928960118 admin admin] PATH_INFO: `/v3/users/70950b4927b14801b6af87eddb71dfa4` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:12.100952 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede6b190-ea59-4590-ad82-ca1928960118 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70950b4927b14801b6af87eddb71dfa4)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:12.102934 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede6b190-ea59-4590-ad82-ca1928960118 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:12.149035 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ede6b190-ea59-4590-ad82-ca1928960118 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70950b4927b14801b6af87eddb71dfa4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:12.150173 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:48:12.162801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ede6b190-ea59-4590-ad82-ca1928960118 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70950b4927b14801b6af87eddb71dfa4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:12.163483 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ede6b190-ea59-4590-ad82-ca1928960118 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70950b4927b14801b6af87eddb71dfa4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:12.171034 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ede6b190-ea59-4590-ad82-ca1928960118 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70950b4927b14801b6af87eddb71dfa4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:12.171717 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ede6b190-ea59-4590-ad82-ca1928960118 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70950b4927b14801b6af87eddb71dfa4'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:12.183969 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1448/2878] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:48:12 2022] DELETE /identity/v3/users/70950b4927b14801b6af87eddb71dfa4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:48:12.189609 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fc6f0-5f12-491f-bb3d-1922c77e6d7a None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:12.189900 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fc6f0-5f12-491f-bb3d-1922c77e6d7a None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:12.190184 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fc6f0-5f12-491f-bb3d-1922c77e6d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:12.208114 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-a60fc6f0-5f12-491f-bb3d-1922c77e6d7a None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:48:12.209331 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a60fc6f0-5f12-491f-bb3d-1922c77e6d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:12.267059 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1431/2879] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:48:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:48:12.276788 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede6b190-ea59-4590-ad82-ca1928960118 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:12.278230 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ede6b190-ea59-4590-ad82-ca1928960118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:12.353376 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede6b190-ea59-4590-ad82-ca1928960118 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:12.358147 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39141903-5129-4f51-a518-afeab8cecee1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:12.358581 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39141903-5129-4f51-a518-afeab8cecee1 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:12.358871 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39141903-5129-4f51-a518-afeab8cecee1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:12.359153 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39141903-5129-4f51-a518-afeab8cecee1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:12.368164 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39141903-5129-4f51-a518-afeab8cecee1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:12.369624 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39141903-5129-4f51-a518-afeab8cecee1 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:12.379035 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1449/2880] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:48:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:48:12.655229 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60fc6f0-5f12-491f-bb3d-1922c77e6d7a None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:12.656693 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-a60fc6f0-5f12-491f-bb3d-1922c77e6d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:12.674675 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60fc6f0-5f12-491f-bb3d-1922c77e6d7a None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:12.679491 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed200bd-526a-4528-b3d8-ff02f903137b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:12.680300 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed200bd-526a-4528-b3d8-ff02f903137b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:12.680580 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed200bd-526a-4528-b3d8-ff02f903137b admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:12.680865 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed200bd-526a-4528-b3d8-ff02f903137b admin admin] PATH_INFO: `/v3/projects/61d12ec947a44a108f8eeab409e3783e` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:12.682105 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed200bd-526a-4528-b3d8-ff02f903137b admin admin] RBAC: Authorizing `identity:delete_project(project_id=61d12ec947a44a108f8eeab409e3783e)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:12.684037 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed200bd-526a-4528-b3d8-ff02f903137b admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:12.730070 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bed200bd-526a-4528-b3d8-ff02f903137b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61d12ec947a44a108f8eeab409e3783e', 'request_id': 'req-bed200bd-526a-4528-b3d8-ff02f903137b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:12.736731 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bed200bd-526a-4528-b3d8-ff02f903137b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61d12ec947a44a108f8eeab409e3783e', 'request_id': 'req-bed200bd-526a-4528-b3d8-ff02f903137b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:12.742203 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bed200bd-526a-4528-b3d8-ff02f903137b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61d12ec947a44a108f8eeab409e3783e', 'request_id': 'req-bed200bd-526a-4528-b3d8-ff02f903137b'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:12.754945 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bed200bd-526a-4528-b3d8-ff02f903137b admin admin] The token cache is being invalidate because project 61d12ec947a44a108f8eeab409e3783e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:48:12.755235 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bed200bd-526a-4528-b3d8-ff02f903137b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:12.755858 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-bed200bd-526a-4528-b3d8-ff02f903137b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:12.757474 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1432/2881] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:48:12 2022] DELETE /identity/v3/projects/61d12ec947a44a108f8eeab409e3783e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:48:12.761729 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39141903-5129-4f51-a518-afeab8cecee1 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:12.771724 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39141903-5129-4f51-a518-afeab8cecee1 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:12.829883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af043ef-3e42-4d0f-83df-a67a70adb8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:12.830762 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af043ef-3e42-4d0f-83df-a67a70adb8f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:12.831051 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af043ef-3e42-4d0f-83df-a67a70adb8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:12.831335 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af043ef-3e42-4d0f-83df-a67a70adb8f7 admin admin] PATH_INFO: `/v3/users/101c25f34eb449e6afee6dbb402cb23b` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:12.832526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af043ef-3e42-4d0f-83df-a67a70adb8f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=101c25f34eb449e6afee6dbb402cb23b)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:12.834502 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af043ef-3e42-4d0f-83df-a67a70adb8f7 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:12.881689 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2af043ef-3e42-4d0f-83df-a67a70adb8f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '101c25f34eb449e6afee6dbb402cb23b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:12.893868 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2af043ef-3e42-4d0f-83df-a67a70adb8f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '101c25f34eb449e6afee6dbb402cb23b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:12.894525 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2af043ef-3e42-4d0f-83df-a67a70adb8f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '101c25f34eb449e6afee6dbb402cb23b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:12.898967 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:48:12.902930 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2af043ef-3e42-4d0f-83df-a67a70adb8f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '101c25f34eb449e6afee6dbb402cb23b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:12.903670 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-2af043ef-3e42-4d0f-83df-a67a70adb8f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '101c25f34eb449e6afee6dbb402cb23b'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:12.917950 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1450/2882] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:48:12 2022] DELETE /identity/v3/users/101c25f34eb449e6afee6dbb402cb23b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:48:13.190172 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed200bd-526a-4528-b3d8-ff02f903137b admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:13.191716 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bed200bd-526a-4528-b3d8-ff02f903137b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:13.206338 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed200bd-526a-4528-b3d8-ff02f903137b admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:13.226307 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:48:13.273854 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695f344-cd75-4ba8-9af6-70b1b90a3995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:13.274741 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695f344-cd75-4ba8-9af6-70b1b90a3995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:13.275031 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695f344-cd75-4ba8-9af6-70b1b90a3995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:13.275314 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695f344-cd75-4ba8-9af6-70b1b90a3995 admin admin] PATH_INFO: `/v3/projects/07fc67e82cfa42d4a5ede0102bcd481c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:13.276522 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695f344-cd75-4ba8-9af6-70b1b90a3995 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07fc67e82cfa42d4a5ede0102bcd481c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:13.278490 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695f344-cd75-4ba8-9af6-70b1b90a3995 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:13.324205 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5695f344-cd75-4ba8-9af6-70b1b90a3995 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07fc67e82cfa42d4a5ede0102bcd481c', 'request_id': 'req-5695f344-cd75-4ba8-9af6-70b1b90a3995'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:13.331526 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5695f344-cd75-4ba8-9af6-70b1b90a3995 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07fc67e82cfa42d4a5ede0102bcd481c', 'request_id': 'req-5695f344-cd75-4ba8-9af6-70b1b90a3995'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:13.337536 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5695f344-cd75-4ba8-9af6-70b1b90a3995 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07fc67e82cfa42d4a5ede0102bcd481c', 'request_id': 'req-5695f344-cd75-4ba8-9af6-70b1b90a3995'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:13.350303 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5695f344-cd75-4ba8-9af6-70b1b90a3995 admin admin] The token cache is being invalidate because project 07fc67e82cfa42d4a5ede0102bcd481c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:48:13.350591 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5695f344-cd75-4ba8-9af6-70b1b90a3995 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:13.351202 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-5695f344-cd75-4ba8-9af6-70b1b90a3995 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:13.352737 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1433/2883] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:48:13 2022] DELETE /identity/v3/projects/07fc67e82cfa42d4a5ede0102bcd481c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:48:13.358157 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a406638-33f2-4984-826c-254cc3af0fde None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:13.358451 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a406638-33f2-4984-826c-254cc3af0fde None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:13.358733 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a406638-33f2-4984-826c-254cc3af0fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:13.376901 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-0a406638-33f2-4984-826c-254cc3af0fde None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:48:13.378146 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-0a406638-33f2-4984-826c-254cc3af0fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:13.438373 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1451/2884] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:48:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:48:13.446103 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab18f7-264a-4746-b22b-7f9847a13415 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:13.446433 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab18f7-264a-4746-b22b-7f9847a13415 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:13.446719 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab18f7-264a-4746-b22b-7f9847a13415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:13.469149 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-49ab18f7-264a-4746-b22b-7f9847a13415 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:48:13.470395 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-49ab18f7-264a-4746-b22b-7f9847a13415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:13.478852 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1434/2885] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:48:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:48:13.483517 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a406638-33f2-4984-826c-254cc3af0fde None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:13.493047 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a406638-33f2-4984-826c-254cc3af0fde None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:13.498638 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754a735-b304-4cb1-95f6-c70249824680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:13.499290 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754a735-b304-4cb1-95f6-c70249824680 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:13.499577 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754a735-b304-4cb1-95f6-c70249824680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:13.499859 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754a735-b304-4cb1-95f6-c70249824680 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:13.500586 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754a735-b304-4cb1-95f6-c70249824680 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:48:13.500895 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754a735-b304-4cb1-95f6-c70249824680 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:13.502532 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754a735-b304-4cb1-95f6-c70249824680 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:13.514856 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1452/2886] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:48:13.519019 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab18f7-264a-4746-b22b-7f9847a13415 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:13.528155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab18f7-264a-4746-b22b-7f9847a13415 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:13.532726 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fb17d-cea5-4f1f-9e4f-b5afb142ded4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:13.533356 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fb17d-cea5-4f1f-9e4f-b5afb142ded4 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:13.533676 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fb17d-cea5-4f1f-9e4f-b5afb142ded4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:13.533952 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fb17d-cea5-4f1f-9e4f-b5afb142ded4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:13.534639 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fb17d-cea5-4f1f-9e4f-b5afb142ded4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:13.536545 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fb17d-cea5-4f1f-9e4f-b5afb142ded4 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:13.561666 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1435/2887] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:48:13 2022] POST /identity/v3/projects => generated 367 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:48:13.565966 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754a735-b304-4cb1-95f6-c70249824680 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:13.576990 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754a735-b304-4cb1-95f6-c70249824680 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:13.632768 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a1916-af66-4cf2-a34d-8d722ef2e205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:13.633428 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a1916-af66-4cf2-a34d-8d722ef2e205 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:13.633745 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a1916-af66-4cf2-a34d-8d722ef2e205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:13.634025 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a1916-af66-4cf2-a34d-8d722ef2e205 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:13.634719 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a1916-af66-4cf2-a34d-8d722ef2e205 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:13.636629 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a1916-af66-4cf2-a34d-8d722ef2e205 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:13.656920 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1453/2888] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:48:13 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:48:13.661003 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fb17d-cea5-4f1f-9e4f-b5afb142ded4 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:13.671725 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fb17d-cea5-4f1f-9e4f-b5afb142ded4 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:13.676247 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce88193-6baf-40cd-810b-6d223aa951f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:13.676887 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce88193-6baf-40cd-810b-6d223aa951f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:13.677168 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce88193-6baf-40cd-810b-6d223aa951f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:13.677441 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce88193-6baf-40cd-810b-6d223aa951f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:13.678201 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce88193-6baf-40cd-810b-6d223aa951f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:13.679777 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce88193-6baf-40cd-810b-6d223aa951f7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:13.690463 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1436/2889] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:48:13.694283 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a1916-af66-4cf2-a34d-8d722ef2e205 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:13.704519 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a1916-af66-4cf2-a34d-8d722ef2e205 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:13.709391 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2c25c-9f39-4597-9f54-f004bbd2267a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:13.710087 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2c25c-9f39-4597-9f54-f004bbd2267a admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:13.710372 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2c25c-9f39-4597-9f54-f004bbd2267a admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:13.710654 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2c25c-9f39-4597-9f54-f004bbd2267a admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:13.711406 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2c25c-9f39-4597-9f54-f004bbd2267a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:13.713009 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2c25c-9f39-4597-9f54-f004bbd2267a admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:13.723656 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1454/2890] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:48:13.727621 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce88193-6baf-40cd-810b-6d223aa951f7 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:13.738553 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce88193-6baf-40cd-810b-6d223aa951f7 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:13.743219 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef61aa-8415-4145-b3fb-de630e07fbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:13.744090 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef61aa-8415-4145-b3fb-de630e07fbdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:13.744379 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef61aa-8415-4145-b3fb-de630e07fbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:13.744668 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef61aa-8415-4145-b3fb-de630e07fbdb admin admin] PATH_INFO: `/v3/projects/b3e1056e0172416d87b84a9def576901/users/31a607a454144aeba2aeda3d608d9374/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:13.759986 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ef61aa-8415-4145-b3fb-de630e07fbdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3e1056e0172416d87b84a9def576901, user_id=31a607a454144aeba2aeda3d608d9374, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:13.762699 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ef61aa-8415-4145-b3fb-de630e07fbdb admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:13.776965 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1437/2891] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:48:13 2022] PUT /identity/v3/projects/b3e1056e0172416d87b84a9def576901/users/31a607a454144aeba2aeda3d608d9374/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:48:13.787326 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea01119-37e7-41f0-ac37-ded3546c98c7 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:13.787844 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea01119-37e7-41f0-ac37-ded3546c98c7 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:13.787952 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea01119-37e7-41f0-ac37-ded3546c98c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:13.815038 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-eea01119-37e7-41f0-ac37-ded3546c98c7 None None] MFA Rules not processed for user `31a607a454144aeba2aeda3d608d9374`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:48:13.816245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-eea01119-37e7-41f0-ac37-ded3546c98c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:13.857753 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:48:13.882606 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1455/2892] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:48:13 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:48:13.887046 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef61aa-8415-4145-b3fb-de630e07fbdb admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:13.933021 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:48:13.952558 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef61aa-8415-4145-b3fb-de630e07fbdb admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:13.957899 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aff2aa-7a9d-4786-bf77-8ad0a7f2137f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:13.958546 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aff2aa-7a9d-4786-bf77-8ad0a7f2137f admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:13.958836 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aff2aa-7a9d-4786-bf77-8ad0a7f2137f admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:13.959117 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aff2aa-7a9d-4786-bf77-8ad0a7f2137f admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:13.959817 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aff2aa-7a9d-4786-bf77-8ad0a7f2137f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:13.961781 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aff2aa-7a9d-4786-bf77-8ad0a7f2137f admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:13.988964 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1438/2893] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:48:13 2022] POST /identity/v3/projects => generated 367 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:48:13.993079 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea01119-37e7-41f0-ac37-ded3546c98c7 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:14.004420 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea01119-37e7-41f0-ac37-ded3546c98c7 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:14.061689 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6bda39-2d44-409c-8e6c-0894d85f4539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:14.062390 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bda39-2d44-409c-8e6c-0894d85f4539 admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:14.062681 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bda39-2d44-409c-8e6c-0894d85f4539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:14.062959 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bda39-2d44-409c-8e6c-0894d85f4539 admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:14.063659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6bda39-2d44-409c-8e6c-0894d85f4539 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:14.065604 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6bda39-2d44-409c-8e6c-0894d85f4539 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:14.085998 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1456/2894] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:48:13 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:48:14.089967 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aff2aa-7a9d-4786-bf77-8ad0a7f2137f admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:14.100305 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aff2aa-7a9d-4786-bf77-8ad0a7f2137f admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:14.104831 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4eacd-ef79-414b-b8a1-44a356784551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:14.105474 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4eacd-ef79-414b-b8a1-44a356784551 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:14.105794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4eacd-ef79-414b-b8a1-44a356784551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:14.106069 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4eacd-ef79-414b-b8a1-44a356784551 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:14.106782 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a4eacd-ef79-414b-b8a1-44a356784551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:14.109016 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a4eacd-ef79-414b-b8a1-44a356784551 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:14.119196 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1439/2895] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:48:14.124430 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6bda39-2d44-409c-8e6c-0894d85f4539 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:14.134571 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6bda39-2d44-409c-8e6c-0894d85f4539 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:14.139393 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b0dc9-e520-42c8-b234-e3049f5042d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:14.140253 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b0dc9-e520-42c8-b234-e3049f5042d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:14.140538 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b0dc9-e520-42c8-b234-e3049f5042d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:14.140818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b0dc9-e520-42c8-b234-e3049f5042d8 admin admin] PATH_INFO: `/v3/projects/168a54a6055e4e0493c4697a9f35bad3/users/9a133ec9070f448db76aff3d856006da/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:14.155822 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b0dc9-e520-42c8-b234-e3049f5042d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=168a54a6055e4e0493c4697a9f35bad3, user_id=9a133ec9070f448db76aff3d856006da, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:14.158606 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b0dc9-e520-42c8-b234-e3049f5042d8 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:14.172601 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1457/2896] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:48:14 2022] PUT /identity/v3/projects/168a54a6055e4e0493c4697a9f35bad3/users/9a133ec9070f448db76aff3d856006da/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:48:14.176520 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4eacd-ef79-414b-b8a1-44a356784551 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:14.187566 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4eacd-ef79-414b-b8a1-44a356784551 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:14.243885 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28146e19-c9e9-470d-8ebd-766ee3b4ebb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:14.244532 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28146e19-c9e9-470d-8ebd-766ee3b4ebb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:14.244813 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28146e19-c9e9-470d-8ebd-766ee3b4ebb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:14.245091 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28146e19-c9e9-470d-8ebd-766ee3b4ebb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:14.245852 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28146e19-c9e9-470d-8ebd-766ee3b4ebb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:14.248155 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28146e19-c9e9-470d-8ebd-766ee3b4ebb8 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:14.258140 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1440/2897] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:48:14.262801 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b0dc9-e520-42c8-b234-e3049f5042d8 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:14.273260 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b0dc9-e520-42c8-b234-e3049f5042d8 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:14.279956 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70373c29-bb08-4429-89a1-e398590b87fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:14.280840 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70373c29-bb08-4429-89a1-e398590b87fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:14.281126 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70373c29-bb08-4429-89a1-e398590b87fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:14.281411 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70373c29-bb08-4429-89a1-e398590b87fe admin admin] PATH_INFO: `/v3/projects/168a54a6055e4e0493c4697a9f35bad3/users/9a133ec9070f448db76aff3d856006da/roles/0ef3687af20a443281e3cfdceca3597d` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:14.283427 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70373c29-bb08-4429-89a1-e398590b87fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=168a54a6055e4e0493c4697a9f35bad3, user_id=9a133ec9070f448db76aff3d856006da, role_id=0ef3687af20a443281e3cfdceca3597d)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:14.286140 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70373c29-bb08-4429-89a1-e398590b87fe admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:14.299410 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1458/2898] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:48:14 2022] PUT /identity/v3/projects/168a54a6055e4e0493c4697a9f35bad3/users/9a133ec9070f448db76aff3d856006da/roles/0ef3687af20a443281e3cfdceca3597d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:48:14.311051 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b87ac-b2de-48ca-a62f-4c19f2b2bbd0 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:14.311337 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b87ac-b2de-48ca-a62f-4c19f2b2bbd0 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:14.311614 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b87ac-b2de-48ca-a62f-4c19f2b2bbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:14.337893 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-4d8b87ac-b2de-48ca-a62f-4c19f2b2bbd0 None None] MFA Rules not processed for user `9a133ec9070f448db76aff3d856006da`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:48:14.339128 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4d8b87ac-b2de-48ca-a62f-4c19f2b2bbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:14.407912 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1441/2899] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:48:14 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:48:14.419066 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70373c29-bb08-4429-89a1-e398590b87fe admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:14.422002 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-70373c29-bb08-4429-89a1-e398590b87fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:14.478262 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70373c29-bb08-4429-89a1-e398590b87fe admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:14.482960 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8543571-e2fb-4307-b400-098c94a2c403 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:14.483696 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8543571-e2fb-4307-b400-098c94a2c403 service cinder] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:14.483986 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8543571-e2fb-4307-b400-098c94a2c403 service cinder] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:14.484268 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8543571-e2fb-4307-b400-098c94a2c403 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:14.493843 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8543571-e2fb-4307-b400-098c94a2c403 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:14.495284 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8543571-e2fb-4307-b400-098c94a2c403 service cinder] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:14.505143 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1459/2900] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:48:14 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:48:14.582298 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b87ac-b2de-48ca-a62f-4c19f2b2bbd0 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:14.592432 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b87ac-b2de-48ca-a62f-4c19f2b2bbd0 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:14.596007 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a326bcc-b8f2-4b94-8d76-4bd2f669c74d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:14.596639 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a326bcc-b8f2-4b94-8d76-4bd2f669c74d service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:14.596917 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a326bcc-b8f2-4b94-8d76-4bd2f669c74d service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:14.597188 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a326bcc-b8f2-4b94-8d76-4bd2f669c74d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:14.631707 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:48:14.658612 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a326bcc-b8f2-4b94-8d76-4bd2f669c74d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:14.660296 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a326bcc-b8f2-4b94-8d76-4bd2f669c74d service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:14.670042 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1442/2901] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:48:14 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:48:19.293122 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8543571-e2fb-4307-b400-098c94a2c403 service cinder] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:19.333622 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:48:19.355980 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8543571-e2fb-4307-b400-098c94a2c403 service cinder] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:19.361630 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781194a9-4fe6-49ac-9a12-7fff89b49f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:19.362496 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781194a9-4fe6-49ac-9a12-7fff89b49f21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:19.362780 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781194a9-4fe6-49ac-9a12-7fff89b49f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:19.363060 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781194a9-4fe6-49ac-9a12-7fff89b49f21 admin admin] PATH_INFO: `/v3/users/31a607a454144aeba2aeda3d608d9374` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:19.364245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781194a9-4fe6-49ac-9a12-7fff89b49f21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31a607a454144aeba2aeda3d608d9374)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:19.366587 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781194a9-4fe6-49ac-9a12-7fff89b49f21 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:19.413622 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-781194a9-4fe6-49ac-9a12-7fff89b49f21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31a607a454144aeba2aeda3d608d9374'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:19.426221 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-781194a9-4fe6-49ac-9a12-7fff89b49f21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31a607a454144aeba2aeda3d608d9374'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:19.426891 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-781194a9-4fe6-49ac-9a12-7fff89b49f21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31a607a454144aeba2aeda3d608d9374'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:19.433558 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-781194a9-4fe6-49ac-9a12-7fff89b49f21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31a607a454144aeba2aeda3d608d9374'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:19.434218 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-781194a9-4fe6-49ac-9a12-7fff89b49f21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31a607a454144aeba2aeda3d608d9374'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:19.446820 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1460/2902] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:48:19 2022] DELETE /identity/v3/users/31a607a454144aeba2aeda3d608d9374 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:48:19.451333 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615ab6b-9823-4f4c-bd3d-5dceea929765 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:19.451793 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615ab6b-9823-4f4c-bd3d-5dceea929765 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:19.451949 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615ab6b-9823-4f4c-bd3d-5dceea929765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:19.469028 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-2615ab6b-9823-4f4c-bd3d-5dceea929765 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:48:19.470275 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2615ab6b-9823-4f4c-bd3d-5dceea929765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:19.529250 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1443/2903] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:48:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:48:19.538583 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781194a9-4fe6-49ac-9a12-7fff89b49f21 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:19.540060 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-781194a9-4fe6-49ac-9a12-7fff89b49f21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:19.598593 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781194a9-4fe6-49ac-9a12-7fff89b49f21 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:19.602308 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df08220-d14d-4783-998a-011d570e2df1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:19.602954 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df08220-d14d-4783-998a-011d570e2df1 service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:19.603242 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df08220-d14d-4783-998a-011d570e2df1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:19.603526 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df08220-d14d-4783-998a-011d570e2df1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:19.613307 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df08220-d14d-4783-998a-011d570e2df1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:19.614778 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df08220-d14d-4783-998a-011d570e2df1 service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:19.624322 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1461/2904] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:48:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:48:19.890617 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615ab6b-9823-4f4c-bd3d-5dceea929765 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:19.892126 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-2615ab6b-9823-4f4c-bd3d-5dceea929765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:19.906158 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615ab6b-9823-4f4c-bd3d-5dceea929765 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:19.910833 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e8630-79aa-45d2-8f75-3ffadaa30371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:19.911695 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e8630-79aa-45d2-8f75-3ffadaa30371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:19.911977 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e8630-79aa-45d2-8f75-3ffadaa30371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:19.912256 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e8630-79aa-45d2-8f75-3ffadaa30371 admin admin] PATH_INFO: `/v3/projects/b3e1056e0172416d87b84a9def576901` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:19.913452 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e8630-79aa-45d2-8f75-3ffadaa30371 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3e1056e0172416d87b84a9def576901)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:19.915445 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e8630-79aa-45d2-8f75-3ffadaa30371 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:19.963192 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1c8e8630-79aa-45d2-8f75-3ffadaa30371 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3e1056e0172416d87b84a9def576901', 'request_id': 'req-1c8e8630-79aa-45d2-8f75-3ffadaa30371'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:19.970808 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1c8e8630-79aa-45d2-8f75-3ffadaa30371 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3e1056e0172416d87b84a9def576901', 'request_id': 'req-1c8e8630-79aa-45d2-8f75-3ffadaa30371'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:19.976515 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1c8e8630-79aa-45d2-8f75-3ffadaa30371 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3e1056e0172416d87b84a9def576901', 'request_id': 'req-1c8e8630-79aa-45d2-8f75-3ffadaa30371'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:19.989442 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1c8e8630-79aa-45d2-8f75-3ffadaa30371 admin admin] The token cache is being invalidate because project b3e1056e0172416d87b84a9def576901 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:48:19.989767 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1c8e8630-79aa-45d2-8f75-3ffadaa30371 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:19.990380 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-1c8e8630-79aa-45d2-8f75-3ffadaa30371 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:19.991910 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1444/2905] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:48:19 2022] DELETE /identity/v3/projects/b3e1056e0172416d87b84a9def576901 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:48:19.995847 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df08220-d14d-4783-998a-011d570e2df1 service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:20.005498 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df08220-d14d-4783-998a-011d570e2df1 service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:20.062800 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05895e2-f8a1-4f35-a445-5d0d94d21fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:20.063731 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05895e2-f8a1-4f35-a445-5d0d94d21fb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:20.063966 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05895e2-f8a1-4f35-a445-5d0d94d21fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:20.064245 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05895e2-f8a1-4f35-a445-5d0d94d21fb1 admin admin] PATH_INFO: `/v3/users/9a133ec9070f448db76aff3d856006da` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:20.065442 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05895e2-f8a1-4f35-a445-5d0d94d21fb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a133ec9070f448db76aff3d856006da)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:20.067460 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05895e2-f8a1-4f35-a445-5d0d94d21fb1 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:20.069027 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:48:20.115418 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d05895e2-f8a1-4f35-a445-5d0d94d21fb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a133ec9070f448db76aff3d856006da'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:20.127553 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d05895e2-f8a1-4f35-a445-5d0d94d21fb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a133ec9070f448db76aff3d856006da'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:20.128219 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d05895e2-f8a1-4f35-a445-5d0d94d21fb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a133ec9070f448db76aff3d856006da'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:20.134996 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d05895e2-f8a1-4f35-a445-5d0d94d21fb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a133ec9070f448db76aff3d856006da'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:20.135626 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-d05895e2-f8a1-4f35-a445-5d0d94d21fb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a133ec9070f448db76aff3d856006da'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:20.148220 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1462/2906] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:48:19 2022] DELETE /identity/v3/users/9a133ec9070f448db76aff3d856006da => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:48:20.422773 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e8630-79aa-45d2-8f75-3ffadaa30371 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:20.424297 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-1c8e8630-79aa-45d2-8f75-3ffadaa30371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:20.438283 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e8630-79aa-45d2-8f75-3ffadaa30371 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:20.502311 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d2c45-45f6-415f-8977-aba9e836c6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:20.503177 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d2c45-45f6-415f-8977-aba9e836c6f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:20.503462 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d2c45-45f6-415f-8977-aba9e836c6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:20.503738 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d2c45-45f6-415f-8977-aba9e836c6f0 admin admin] PATH_INFO: `/v3/projects/168a54a6055e4e0493c4697a9f35bad3` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:20.504919 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d2c45-45f6-415f-8977-aba9e836c6f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=168a54a6055e4e0493c4697a9f35bad3)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:20.506903 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d2c45-45f6-415f-8977-aba9e836c6f0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:20.508353 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:48:20.557424 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-909d2c45-45f6-415f-8977-aba9e836c6f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '168a54a6055e4e0493c4697a9f35bad3', 'request_id': 'req-909d2c45-45f6-415f-8977-aba9e836c6f0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:20.563521 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-909d2c45-45f6-415f-8977-aba9e836c6f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '168a54a6055e4e0493c4697a9f35bad3', 'request_id': 'req-909d2c45-45f6-415f-8977-aba9e836c6f0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:20.569078 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-909d2c45-45f6-415f-8977-aba9e836c6f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '168a54a6055e4e0493c4697a9f35bad3', 'request_id': 'req-909d2c45-45f6-415f-8977-aba9e836c6f0'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:20.581794 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-909d2c45-45f6-415f-8977-aba9e836c6f0 admin admin] The token cache is being invalidate because project 168a54a6055e4e0493c4697a9f35bad3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:48:20.582018 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-909d2c45-45f6-415f-8977-aba9e836c6f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:20.582632 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-909d2c45-45f6-415f-8977-aba9e836c6f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:20.584181 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1445/2907] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:48:20 2022] DELETE /identity/v3/projects/168a54a6055e4e0493c4697a9f35bad3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:48:20.590007 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b6de7-c3a4-4c53-b78e-7705fbb663c2 None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:20.590572 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b6de7-c3a4-4c53-b78e-7705fbb663c2 None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:20.590682 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b6de7-c3a4-4c53-b78e-7705fbb663c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:20.610952 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-bb0b6de7-c3a4-4c53-b78e-7705fbb663c2 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:48:20.612198 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-bb0b6de7-c3a4-4c53-b78e-7705fbb663c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:20.675164 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1463/2908] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:48:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:48:20.683161 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2924901-b807-4e64-871e-a04e194d629b None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:20.683460 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2924901-b807-4e64-871e-a04e194d629b None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:20.683749 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2924901-b807-4e64-871e-a04e194d629b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:20.703072 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-e2924901-b807-4e64-871e-a04e194d629b None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:48:20.704279 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-e2924901-b807-4e64-871e-a04e194d629b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:20.712270 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1446/2909] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:48:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:48:20.716541 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b6de7-c3a4-4c53-b78e-7705fbb663c2 None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:20.726112 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b6de7-c3a4-4c53-b78e-7705fbb663c2 None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:20.730603 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4041b-d6a6-46d2-8034-79217b029217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:20.731272 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4041b-d6a6-46d2-8034-79217b029217 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:20.731576 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4041b-d6a6-46d2-8034-79217b029217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:20.731896 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4041b-d6a6-46d2-8034-79217b029217 admin admin] PATH_INFO: `/v3/domains` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:20.732707 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d4041b-d6a6-46d2-8034-79217b029217 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80207) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:48:20.733013 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d4041b-d6a6-46d2-8034-79217b029217 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:20.734653 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d4041b-d6a6-46d2-8034-79217b029217 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:20.745883 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1464/2910] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 30 18:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:48:20.750696 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2924901-b807-4e64-871e-a04e194d629b None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:20.759927 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2924901-b807-4e64-871e-a04e194d629b None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:20.765508 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc3594-6dff-4f8a-b563-e9c3bb70cd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:20.766182 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc3594-6dff-4f8a-b563-e9c3bb70cd24 admin admin] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:20.766471 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc3594-6dff-4f8a-b563-e9c3bb70cd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:20.766752 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc3594-6dff-4f8a-b563-e9c3bb70cd24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:20.767487 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc3594-6dff-4f8a-b563-e9c3bb70cd24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:20.769390 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc3594-6dff-4f8a-b563-e9c3bb70cd24 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:20.796221 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1447/2911] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 30 18:48:20 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:48:20.800263 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4041b-d6a6-46d2-8034-79217b029217 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:20.810434 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4041b-d6a6-46d2-8034-79217b029217 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:20.868495 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d907c-626a-4a02-8371-2f7638109e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:20.869082 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d907c-626a-4a02-8371-2f7638109e8f admin admin] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:20.869370 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d907c-626a-4a02-8371-2f7638109e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:20.869685 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d907c-626a-4a02-8371-2f7638109e8f admin admin] PATH_INFO: `/v3/users` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:20.870374 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d907c-626a-4a02-8371-2f7638109e8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:20.872762 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d907c-626a-4a02-8371-2f7638109e8f admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:20.882116 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:48:20.896071 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1465/2912] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 30 18:48:20 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:48:20.897323 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc3594-6dff-4f8a-b563-e9c3bb70cd24 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:20.908474 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc3594-6dff-4f8a-b563-e9c3bb70cd24 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:20.913127 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79f1ec-9ecb-4515-a5e7-2a3045e48fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:20.913815 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79f1ec-9ecb-4515-a5e7-2a3045e48fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:20.914131 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79f1ec-9ecb-4515-a5e7-2a3045e48fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:20.914408 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79f1ec-9ecb-4515-a5e7-2a3045e48fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:20.915160 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a79f1ec-9ecb-4515-a5e7-2a3045e48fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:20.916760 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a79f1ec-9ecb-4515-a5e7-2a3045e48fa0 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:20.927823 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1448/2913] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:48:20.931622 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d907c-626a-4a02-8371-2f7638109e8f admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:20.941635 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d907c-626a-4a02-8371-2f7638109e8f admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:20.946549 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca2046-c731-489a-96b7-862735503a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:20.947206 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca2046-c731-489a-96b7-862735503a26 admin admin] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:20.947494 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca2046-c731-489a-96b7-862735503a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:20.947772 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca2046-c731-489a-96b7-862735503a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:20.948495 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca2046-c731-489a-96b7-862735503a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:20.950149 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca2046-c731-489a-96b7-862735503a26 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:20.960152 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1466/2914] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 30 18:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:48:20.964180 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79f1ec-9ecb-4515-a5e7-2a3045e48fa0 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:20.975285 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79f1ec-9ecb-4515-a5e7-2a3045e48fa0 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:20.980313 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde90fd-e65e-41d5-a401-53ffac982c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:20.981175 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde90fd-e65e-41d5-a401-53ffac982c46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:20.981457 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde90fd-e65e-41d5-a401-53ffac982c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:20.982342 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde90fd-e65e-41d5-a401-53ffac982c46 admin admin] PATH_INFO: `/v3/projects/8de1de27fe724263921f7049c6a7ab36/users/efc928748f214c248e2686d7389e130f/roles/3fbb71ae2de04e78b68d6d6f407ef09c` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:20.996434 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde90fd-e65e-41d5-a401-53ffac982c46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8de1de27fe724263921f7049c6a7ab36, user_id=efc928748f214c248e2686d7389e130f, role_id=3fbb71ae2de04e78b68d6d6f407ef09c)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:20.999624 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde90fd-e65e-41d5-a401-53ffac982c46 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:21.011552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1449/2915] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 30 18:48:20 2022] PUT /identity/v3/projects/8de1de27fe724263921f7049c6a7ab36/users/efc928748f214c248e2686d7389e130f/roles/3fbb71ae2de04e78b68d6d6f407ef09c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:48:21.022637 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dec80-66c6-48ad-85d0-a8a40fd5765d None None] REQUEST_METHOD: `POST` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:21.022933 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dec80-66c6-48ad-85d0-a8a40fd5765d None None] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:21.023216 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dec80-66c6-48ad-85d0-a8a40fd5765d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:21.050453 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.auth.core [None req-730dec80-66c6-48ad-85d0-a8a40fd5765d None None] MFA Rules not processed for user `efc928748f214c248e2686d7389e130f`. Rule list: `[]` (Enabled: `True`). {{(pid=80207) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:48:21.051689 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-730dec80-66c6-48ad-85d0-a8a40fd5765d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:21.115408 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1467/2916] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:48:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:48:21.126609 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde90fd-e65e-41d5-a401-53ffac982c46 admin admin] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:21.128276 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-4cde90fd-e65e-41d5-a401-53ffac982c46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:21.184252 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde90fd-e65e-41d5-a401-53ffac982c46 admin admin] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:21.188511 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee316757-b6e2-4714-a4f6-c8eb3eb74c38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92c70ce830ea4dd981716d622b16d595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:21.189143 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee316757-b6e2-4714-a4f6-c8eb3eb74c38 service cinder] REQUEST_METHOD: `GET` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:21.189429 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee316757-b6e2-4714-a4f6-c8eb3eb74c38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:21.189742 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee316757-b6e2-4714-a4f6-c8eb3eb74c38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:21.198010 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee316757-b6e2-4714-a4f6-c8eb3eb74c38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:21.199446 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee316757-b6e2-4714-a4f6-c8eb3eb74c38 service cinder] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:21.208310 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1450/2917] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:48:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:48:29.747897 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730dec80-66c6-48ad-85d0-a8a40fd5765d None None] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:29.811252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730dec80-66c6-48ad-85d0-a8a40fd5765d None None] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:29.815823 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c0bad-d2fb-4e7c-8a19-b14c0ff6f0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:29.816685 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c0bad-d2fb-4e7c-8a19-b14c0ff6f0b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:29.816970 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c0bad-d2fb-4e7c-8a19-b14c0ff6f0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:29.817252 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c0bad-d2fb-4e7c-8a19-b14c0ff6f0b0 admin admin] PATH_INFO: `/v3/users/efc928748f214c248e2686d7389e130f` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:29.818466 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c0bad-d2fb-4e7c-8a19-b14c0ff6f0b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efc928748f214c248e2686d7389e130f)` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:29.820386 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c0bad-d2fb-4e7c-8a19-b14c0ff6f0b0 admin admin] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:29.864166 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ec1c0bad-d2fb-4e7c-8a19-b14c0ff6f0b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efc928748f214c248e2686d7389e130f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:29.876648 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ec1c0bad-d2fb-4e7c-8a19-b14c0ff6f0b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efc928748f214c248e2686d7389e130f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:29.877403 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ec1c0bad-d2fb-4e7c-8a19-b14c0ff6f0b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efc928748f214c248e2686d7389e130f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:29.884162 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ec1c0bad-d2fb-4e7c-8a19-b14c0ff6f0b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efc928748f214c248e2686d7389e130f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:29.884793 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.notifications [None req-ec1c0bad-d2fb-4e7c-8a19-b14c0ff6f0b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efc928748f214c248e2686d7389e130f'} {{(pid=80207) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:29.887241 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:48:29.898840 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1468/2918] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 30 18:48:29 2022] DELETE /identity/v3/users/efc928748f214c248e2686d7389e130f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:48:29.903252 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb3a70-fa1c-4e6f-9326-5e2548d179e1 None None] REQUEST_METHOD: `POST` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:29.903756 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb3a70-fa1c-4e6f-9326-5e2548d179e1 None None] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:29.903872 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb3a70-fa1c-4e6f-9326-5e2548d179e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:29.920733 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.auth.core [None req-bddb3a70-fa1c-4e6f-9326-5e2548d179e1 None None] MFA Rules not processed for user `de745ad7f2ed48ffbb043529099e53f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80206) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:48:29.921942 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bddb3a70-fa1c-4e6f-9326-5e2548d179e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:29.948290 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:48:29.985211 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1451/2919] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 30 18:48:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:48:29.994704 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c0bad-d2fb-4e7c-8a19-b14c0ff6f0b0 admin admin] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:29.996109 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-ec1c0bad-d2fb-4e7c-8a19-b14c0ff6f0b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:30.054566 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c0bad-d2fb-4e7c-8a19-b14c0ff6f0b0 admin admin] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:30.058202 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b306a9b-a986-4be2-9eae-87d948915c5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d8c54102fb340859ed0acc3189fe376', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:30.058842 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b306a9b-a986-4be2-9eae-87d948915c5d service neutron] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:30.059129 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b306a9b-a986-4be2-9eae-87d948915c5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:30.059409 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b306a9b-a986-4be2-9eae-87d948915c5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:30.069406 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b306a9b-a986-4be2-9eae-87d948915c5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:30.070818 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b306a9b-a986-4be2-9eae-87d948915c5d service neutron] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:30.081101 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1469/2920] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:48:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:48:30.348482 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb3a70-fa1c-4e6f-9326-5e2548d179e1 None None] Authenticating user token {{(pid=80206) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:48:30.349937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.fernet_utils [None req-bddb3a70-fa1c-4e6f-9326-5e2548d179e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80206) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:48:30.362700 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb3a70-fa1c-4e6f-9326-5e2548d179e1 None None] Validating token access rules against request {{(pid=80206) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:48:30.367550 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aaa588-d0f0-4bc8-b087-9f348f944af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de745ad7f2ed48ffbb043529099e53f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed5e6c375de4a6186ba026da89db56b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80206) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:48:30.368564 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aaa588-d0f0-4bc8-b087-9f348f944af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:48:30.368919 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aaa588-d0f0-4bc8-b087-9f348f944af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:48:30.369208 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aaa588-d0f0-4bc8-b087-9f348f944af7 admin admin] PATH_INFO: `/v3/projects/8de1de27fe724263921f7049c6a7ab36` {{(pid=80206) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:48:30.370452 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aaa588-d0f0-4bc8-b087-9f348f944af7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8de1de27fe724263921f7049c6a7ab36)` {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:48:30.372377 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aaa588-d0f0-4bc8-b087-9f348f944af7 admin admin] RBAC: Authorization granted {{(pid=80206) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:48:30.421911 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e1aaa588-d0f0-4bc8-b087-9f348f944af7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8de1de27fe724263921f7049c6a7ab36', 'request_id': 'req-e1aaa588-d0f0-4bc8-b087-9f348f944af7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:30.427937 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e1aaa588-d0f0-4bc8-b087-9f348f944af7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8de1de27fe724263921f7049c6a7ab36', 'request_id': 'req-e1aaa588-d0f0-4bc8-b087-9f348f944af7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:30.433552 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e1aaa588-d0f0-4bc8-b087-9f348f944af7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8de1de27fe724263921f7049c6a7ab36', 'request_id': 'req-e1aaa588-d0f0-4bc8-b087-9f348f944af7'} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:30.445679 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e1aaa588-d0f0-4bc8-b087-9f348f944af7 admin admin] The token cache is being invalidate because project 8de1de27fe724263921f7049c6a7ab36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80206) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:48:30.445964 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e1aaa588-d0f0-4bc8-b087-9f348f944af7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:30.446618 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG keystone.notifications [None req-e1aaa588-d0f0-4bc8-b087-9f348f944af7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80206) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:48:30.448173 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: [pid: 80206|app: 0|req: 1452/2921] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 30 18:48:30 2022] DELETE /identity/v3/projects/8de1de27fe724263921f7049c6a7ab36 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:48:40.075990 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:48:40.442792 dv-cinder-803785-1-iscsi devstack@keystone.service[80206]: DEBUG dbcounter [-] [80206] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80206) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:52:53.377659 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b306a9b-a986-4be2-9eae-87d948915c5d service neutron] Authenticating user token {{(pid=80207) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:52:53.379105 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-3b306a9b-a986-4be2-9eae-87d948915c5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:52:53.518928 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b306a9b-a986-4be2-9eae-87d948915c5d service neutron] Validating token access rules against request {{(pid=80207) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:52:53.579886 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936aea9c-412f-4867-8116-b2e8008fd5fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49325c1733174e0c98b3475a10c9b5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c7e1ee12aa14b7ab13e07822826b25b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80207) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:52:53.580529 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936aea9c-412f-4867-8116-b2e8008fd5fd service placement] REQUEST_METHOD: `GET` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:52:53.580841 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936aea9c-412f-4867-8116-b2e8008fd5fd service placement] SCRIPT_NAME: `/identity` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:52:53.581118 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936aea9c-412f-4867-8116-b2e8008fd5fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80207) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:52:53.583515 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.fernet_utils [None req-936aea9c-412f-4867-8116-b2e8008fd5fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80207) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:52:53.594603 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936aea9c-412f-4867-8116-b2e8008fd5fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:52:53.596168 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936aea9c-412f-4867-8116-b2e8008fd5fd service placement] RBAC: Authorization granted {{(pid=80207) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:52:53.606021 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: [pid: 80207|app: 0|req: 1470/2922] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 30 18:52:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:53:03.601705 dv-cinder-803785-1-iscsi devstack@keystone.service[80207]: DEBUG dbcounter [-] [80207] Writing DB stats keystone:SELECT=89 {{(pid=80207) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}